1 GIScience and the Big Data Age Yihong Yuan Department of Geography Texas State University

1

GIScience and the Big Data Age

Yihong Yuan

Department of Geography

Texas State University 

2

About me

• Yihong YuanAssistant Professor

yuan@txstate.edu. ELA 366, 512-245-3208

• Research Interests– Spatio-temporal data mining– Human mobility and activity patterns– Big data analytics

3

Geography and Big Data

• GIS– Not only about mapping functions

• Big Geo-data– Information and communication technologies

(ICTs)• Greater mobility flexibility• A wide range of spatio-temporal data sources• Align marketing campaigns to spatial patterns.

4

• “Geography is one of the most natural, logical and intuitive ways to discover, visualize, overlay, compare, slice, sort and apply big data to a problem”

• “GIS used to be about the analysis of relatively static

institutional data, but new data streams mean that today’s GIS problems look very much the same as today’s big data problems: extract meaningful information from a fire hose of inputs”

5

• Traditional geographic knowledge discovery– e.g., high resolution trajectories

• Incomplete Spatio-temporal datasets– Low resolution– Few individual attributes– Uncertainty?

6

Past Research• Georeferenced mobile phone data analytics

– Individual-oriented research– Activity space

» Measurements: Radius, Eccentricity, entropy» Correlation between phone usage and activity space

– Trajectory and sequence patterns» Time series analysis

– Urban-oriented studies• Spatial clusters • Spatial rhythms

• Dynamic clustering• Functional time points

7

UML Model about Geo-referenced mobile phone data

Knowledge Discovery Tasks

Generalize types of information in mobile

phone datasets

Urban-oriented research

Construct an UML model

Analyze individual activity space

Individual-oriented research

Measure trajectory similarity

Correlate activity space with phone usage

Correlate activity space with individual and

supra-individual attributes

Identify urban hotspots and clusters

Dynamic clustering and time series analysis

Extract functional time points

8

• Mobile Phone Connections in 10 cities in northeast China– Time, Duration, and Locations of Mobile

Phone Connections in 9 days– Age and Gender Attributes of the Users– Possibility of simulated data

Example Mobile Phone Dataset

9

Analysis of Activity space

• Three measurements– Radius -> Scale

• eigenvectors of trajectories

– Eccentricity -> Shape• Range [0,1]• Closer to a straight line or a circle

– Entropy->Regularity• How random the visiting patterns are

2 2

1

ˆ 1-( )

ˆ

ee

e

2e1e

21

- logN

i ii

E p p

10

Correlation between individual activity space and phone usage

11

Results

• For People with Higher Mobile Phone Usage:  – Larger Activity Space– Trajectories are Closer to a Circle– Movement is More Random, Less

Predictable

12

Activity space vs Trajectory

13

Analysis of trajectory patterns

• Compare trajectories from phone records– Sequences of cell IDs

• Edit distance Method– String matching and auto-correction

14

Analysis of trajectory patterns (Cont.)

• Applications– Identify similar users

• Clustering analysis

– Identify outlier users

15

Knowledge Discovery Tasks

Generalize types of information in mobile

phone datasets

Urban-oriented research

Construct an UML model

Analyze individual activity space

Individual-oriented research

Measure trajectory similarity

Correlate activity space with phone usage

Correlate activity space with individual and

supra-individual attributes

Identify urban hotspots and clusters

Dynamic clustering and time series analysis

Extract functional time points

16

• The changing clustering of urban area

Urban hotspots and clusters

Weekdays Weekends

T2: 2pm-3pm

T1:8am-9am

T3: 7pm-8pm

17

• Mobility patterns of different population groups– Weekday 2pm-3pm

Urban clusters (Cont.)

Age: 12-17 Age: > 60

18

• Provide input for urban infrastructure planning– Are public facilities where people are??

Urban clusters (Cont.)

Age: > 60

A park

19

Dynamic Clustering

• Focus on “rhythms” instead of just “clusters”

• Various mobility patterns in urban area– How to explore? – time series analysis

CBD, Beijing Suburb, Beijing

20

Dynamic Clustering (Cont.)

• Methods– Divide study area

• Voronoi polygon (based on towers)• What to compare: 24-hour series for each

polygon based on mobility count

• Outlier detection e.g., traffic congestion

21

Outlier polygons

• 15 outliers for weekdays and 18 for weekends

Weekday Weekends

22

Mobility patterns in outlier areas• Outlier Polygon 238

– Night clubs and other leisure facilities– International trading center

• Outlier Polygon 125– Several community colleges – Not many night clubs, bars, etc.

Polygon 238 Polygon 125

23

Current and future research

24

Setting up functional time in cities

• Standardization of time – Determination of the beginning/end of a day

• The development of ICT– Real-time activity patterns– More flexibility in time management and

activity scheduling• i.e., fixed parking hour policy may not be

applicable in Central business districts

25

Setting up functional time in cities

26

Cross-country comparison for Social Media websites

• Flickr data, 100 million records and geo-tagged photos

• Similarity and dissimilarity of human mobility in various cities– “A tale of many cities”

27

Current and future research• Mobility patterns in

developing and developed countries– China as a focus

• Weibo and Twitter check-in data– Comparison study for

special time period– Holiday patterns

28

Current and future research

• Mass media and Social Media– GDELT dataset

• Geo-tagged news Events from 1970s

– Public relations and interaction between countries

29

(a) (b)

30

Big data and GIS jobs…• Traditional GIS jobs:• GIS Technician/Analyst/consultant• GIS manager/researcher• ……• Where are the positions?• Public sector… NGA, USGS, State and local Gov, DOT,

planning dept.• Private company…Oil&Gas, Mapping companies, Land

management, Utility…• Non-profit agency… Nature Conservancy, International

Crane Foundation• Consulting firms…Surveying, Remote Sensing…

31

Example: Private Sector Jobs

• Mapping Companies• Software Developers• Utilities• Land Development• Non-Profits• Others

32

Job Skills

• Project Management• Technical Support• Report Writing• Public Speaking• Research/Literature review• Programming

33

Software Skills (cont.)

• GIS software packages• ArcGIS, ENVI, GDAL• Mobile & Web Technology

– Silverlight / Flex /HTML / ASP– Android Dev

• Python / C#...• Database: Access, SQL

Server, PostgresSQL

34

Job Postings• Company Website

– ESRI summer internship program• Relevant Employment Websites

– General sites: Monster.com / Indeed.com– Linkedin.com– Glassdoor.com– GIS Jobs Clearinghouse (gjc.org)– GISjobs.com & Geojobs.org– GeoCommunity – GIS Café – WI State Cartographers Office

• http://www.sco.wisc.edu/jobs/jobs.php

35

Job Postings

• Internal Company Postings• Company Website• Relevant Employment Websites

– GIS Jobs Clearinghouse (gjc.org)– GISjobs.com & Geojobs.org– GeoCommunity – GIS Café – Monster.com

36

Job Postings

• Internal Company Postings• Company Website• Relevant Employment Websites

– GIS Jobs Clearinghouse (gjc.org)– GISjobs.com & Geojobs.org– GeoCommunity – GIS Café – Monster.com

37

Big data jobs…• Spatial data are inherently big data…• For GIS major…

– Data Scientist• This is a more “General” term• Focus on big (geo)data analytics• Highly competitive salary• Graduate degree (MA possible, PhD preferred)• Many opportunities…

• Skill set:• Strong statistical background• Strong and programming: Python, R, etc,

38

Example positions

• Data Scientist @ ESRI– http://www.simplyhired.com/job/data-scientist-agriculture-job/esri/5jjxyxjt4b?cid=n

tvzgigizsvnqhofbuscopqozjkxqugd

• Research Data Scientist– http://www.americasjobexchange.com/job-detail/job-opening-AJE-56966

1132?source=indeed&utm_source=Indeed&utm_medium=cpc&utm_campaign=Indeed

• Other potential groups: Apple geo-group, Twitter geo-group, Facebook data science group

39

Thanks!Questions and Comments?