PyCon PH 2014 - GeoComputation

GEOCOMPUTATIONEngr. Ranel O. Padon

PyCon PH 2014 | [email protected]

http://www.gctours.net/product_images/uploaded_images/grand-canyon-hd720.jpg

Full-Time Drupal Developer (CNN Travel)

Lecturer, UP DGE (Java/Python OOP Undergrad Courses)

Lecturer, UP NEC (Web GIS Training Course)

BS Geodetic Engineering in UP

MS Computer Science in UP (25/30 units)

Involved in Java, Python, and Drupal projects.

ABOUT ME

http://travel.cnn.com/

http://dge.upd.edu.ph/dge/

http://www.upnec.com/trainingprograms/online-geography-information-system-and-web-mapping/

http://www.up.edu.ph/

http://coe.upd.edu.ph/old/index.php?q=dcs/degree/mscs

The role of Python in implementing a rapid and

mass valuation of lots along the Pasig River

tributaries.

This is the story of what we have done.

ABOUT MY TOPIC

I• PRTSAS BACKGROUND

II• VALUATION COMPONENT

III• AHP MODELING

IV• RECOMMENDATIONS

TOPIC FLOW

OF FLOOD AND MEN

http://www.reynaelena.com/wp-content/uploads/2009/09/ondoy-aftermath-by-wenzzo-pancho.jpg

http://1.bp.blogspot.com/-sdUQ_XBc5o8/TnfOuNASgjI/AAAAAAAAAug/u-OQ1Cv5oEg/s1600/Ondoymissionhospital.jpg

http://filsg.com/download/ondoy16.jpg

Man is a complex being:

he makes deserts bloom - and lakes die.

GIL SCOTT-HERON

http://i.dailymail.co.uk/i/pix/2011/05/28/article-0-0C4E40E200000578-673_468x301.jpg

http://d2tq98mqfjyz2l.cloudfront.net/image_cache/1254443971159430.jpeg

PASIG RIVER | BEFORE

http://ourss14blog.blogspot.com/2011/10/article-xii-national-economy-and.html

PASIG RIVER | AFTER

http://ourss14blog.blogspot.com/2011/10/article-xii-national-economy-and.html

PRTSAS = Pasig River Tributaries Survey and Assessment Study

PRTSAS = PRRC + UP TCAGP

Aims to gather baseline information on the physical

characteristics of major and minor tributaries of the Pasig River.

The gathered information will be used to properly manage the

river and correctly steer its rehabilitation.

BACKGROUND | PRTSAS

“To transform

Pasig River

and its environs

into a showcase

of a new quality

of urban life.”

BACKGROUND | PRTSAS | PRRC

http://www.prrc.gov.ph/

Restore the Pasig River to its

historically pristine condition by

applying bio-eco engineering and

attain a sustainable socio-economic

development.

Relocation of formal and informal

settlers.

Regulate the 3-m easement.

BACKGROUND | PRTSAS | PRRC

BACKGROUND | PRTSAS | UP TCAGP

http://dge.upd.edu.ph/dge/about/about-tcagp/

Research and extension arm of UP DGE.

Large-Scale Projects:

DREAM (DOST NOAH)

PRTSAS

PRS 92 R&D and Implementation Support

BACKGROUND | PRTSAS | UP TCAGP

http://noah.dost.gov.ph/

PRTSAS has 5 major components:

Parcel/As-Built Survey

Hydrographic Component

Water Quality/Environmental Impact

Easement and Adjoining Lots Valuation

Web GIS

BACKGROUND | PRTSAS | COMP.

BACKGROUND | PRTSAS | COVERAGE

BACKGROUND | PRTSAS | COVERAGE

To perform individual valuation work of the PRRC proposed

relocation sites.

To perform a rapid appraisal of the 3-meter easements and

adjoining lots for all tributary locations.

To develop and perform an automated GIS-assisted valuation

of the lots adjoining all tributaries.

VALUATION | DUTIES

VALUATION | THE TEAM

Develop a GIS-assisted valuation model and

perform automated valuation of lots

adjoining the tributaries.

VALUATION | OVERVIEW

VALUATION | EASEMENT CONDITION

Fully-Developed

Partially-Developed

Undeveloped/

Depreciated

determined by the highest price a property can command

if put up for sale in an open market

determinations are made from market evidence or

transactions and found on published market listings or

information from market participants.

VALUATION | MARKET VALUE

The ultimate question is: how do you value a land?

And how do you value lands with huge coverage rapidly?

VALUATION | MARKET VALUE

http://blog.melvinpereira.com/wp-content/uploads/2011/04/man-thinking.jpg

http://e.peruthisweek.e3.pe//ima/0/0/0/1/5/15908/624x468.png

AHP Model Formulation

Geospatial Data Buildup

Market Value Geoprocessing

Market Value Map

GENERAL PROCESS FLOW

ArcPy

http://ithelp.port.ac.uk/images/SPSS-logo-32F23C8B51-seeklogo.png

http://www.lic.wisc.edu/training/Images/arcgis.gif

http://www.logilab.org/

Analytic Hierarchy Process is a decision-making method

based on mathematics and psychology developed by Prof.

Thomas L. Saaty in the 1970s.

The input can be obtained from actual measurements such

as price, weight, etc. and from subjective opinion such as

satisfaction feelings and preferences.

AHP

http://www.nae.edu/File.aspx?id=41107

used in scientific and business contexts

useful in situation with scarce, but high-quality or high-

importance data

80/20 Principle: essential information (80%) could be

expressed by just a small but important set of data (20%)

unlike the case of face recognition problem which

requires voluminous data to be stable

AHP

http://www.nae.edu/File.aspx?id=41107

AHP | CHOOSING A LEADER

http://en.wikipedia.org/wiki/Analytic_Hierarchy_Process

AHP | CHOOSING A LEADER

BRAIN

http://en.wikipedia.org/wiki/Analytic_Hierarchy_Process

AHP | CHOOSING A PARTNER

1. Parameters

II. Weights of Parameters

AHP | MURPHY’S LAW OF LOVE

BEAUTY AVAILABILITY

BRAIN

B· B· A = k

AHP | I. PARAMETERS

Intelligence

Values

Humor

Beauty

Wealth

Religion

Health

Interests

Sports

Zodiac Sign

and so on

Choosing a partner

AHP | I. PARAMETERS

Use statistical software to evaluate if some factors

could be eliminated, values to watch out:

1.) Kaiser-Meyer-Olkin (KMO) Coefficient –

tests whether the partial correlations among variables are small

2.) Barlett’s Test for Sphericity (BTS) –

tests whether the correlation matrix is an identity matrix

Choosing a partner

AHP | I. PARAMETERS

Why Dimensionality Reduction?

To simplify data structures

Conserve computing and/or storage resources

Examples: Face Recognition, MP3 and JPEG file formats,

Douglas-Peucker Algorithm

AHP | I. PARAMETERS

Dimensionality Reduction | EigenFaces

Principal vectors used in the problem of human face recognition

http://cognitrn.psych.indiana.edu/nsfgrant/FaceMachine/faceMachine.html

AHP | I. PARAMETERS

Dimensionality Reduction/Factor Analysis

Is the strength of the relationships

among variables large enough?

Is it a good idea to proceed a factor analysis for the data?

Choosing a partner

AHP | II. WEIGHTS OF PARAMETERS

Possible major components after Factor Extraction

1. Humor

2. Beauty

3. Intelligence

Choosing a partner


Sample Preference Matrix (3 Parameters)

Choosing a partner

Criteria More

ImportantIntensity

A B

Humor Beauty A 5

Humor Intelligence A 7

Beauty Intelligence A 3


Choosing a partner


As you might observed, we need to reduce the

number of parameters so that the respondents/evaluators

will just have to evaluate the smallest preference matrix possible.

Choosing a partner

AHP | FINAL PARAMETERS’ WEIGTHS

Apply the AHP algorithm to compute the relative weights,

possible result:

0.60 Humor

0.25 Beauty

0.15 Intelligence

Choosing a partner

AHP | FINAL PARAMETERS’ WEIGTHS

Optimum Partner (among alternatives/suitors)

= 0.60 Humor + 0.25 Beauty + 0.15 Intelligence

Choosing a partner

AHP | VALUING A LAND

1. Parameters

II. Weights of Parameters

III. Weights of Sub-Categories

http://i.domainstatic.com.au/b432bfa9-1e06-4d69-812e-ea14e22d0112/domain/20108120961pio04192711

AHP | I. PARAMETERS

Lot Shape

Topography

Easement Condition

Neighborhood Classification

Accessibility to Main Roads

Corner Influence

Land-Use Type

Proximity to Commercial Area

Proximity to Churches

Proximity to Markets

Proximity to School

Proximity to LGUs

Existing Improvements

Public Utilities

and so on

Obtaining the optimal land value

AHP | I. PARAMETERS

AHP | I. PARAMETERS

We used SPSS for computing the KMO and BTS

Coefficients.

1.) KMO > 0.5

2.) BTS < 0.001

SPSS also provides validation values that could be used

when we decide to automate the process in pure Python later.

Choosing a partner

Factor Analysis (18 raw & unordered variables)

AHP | I. PARAMETERS

Extracted Factors

AHP | I. PARAMETERS

Land-Use

Accessibility

Lot Size

Lot Shape

Neighborhood


Sample Preference Matrix (4 Parameters)

Choosing a car: 4 Params, 6 Comparisons

Criteria More

ImportantIntensity

A B

Cost Safety A 3

Cost Style A 7

Cost Capacity A 3

Safety Style A 9

Safety Capacity A 1

Style Capacity B 7


Actual Data

Obtaining the Optimal Value : 5 Params, 10 Comparisons

The CSV File


AHP Algorithms (Ishizaka & Lusti, 2006)

1. The Eigenvalue Approach (Power Method)

2. The Geometric Mean

3. The Mean of Normalized Values


3. The Mean of Normalized Values




Effective AHP parameters

Parameter Weight

Land Use 0.372

Location/Accessibility 0.276

Lot Size 0.125

Lot Shape 0.111

Neighborhood Classification 0.116


Some issues for the computation of our

AHP parameters:

1.) Assumes all respondents have

consistent preference matrices

2.) Uses the arithmetic mean for computing the

effective parameter weights across

all the respondents.


consistency means that if A>B and B>C then A>C,

where A, B, and C, refer to the criteria/parameters

of the land value.

It also means that if A > 2*B and B > 3*C then A > 6*C,

as the number of criteria increases, it's more difficult

to be consistent


We have implemented the proposed Saaty's

Consistency Measure of the preference matrix of the

respondents but we have found it to be too limiting.


Pelaez and Lamata (2002) proposed a new way of

computing the Consistency Index and that is by using

the concept of determinants.

We implemented their paper using Python and

NumPy and we obtained a better filtering for the

consistent survey answers.




However, [Aragon, et al (2012)], shown that it is

better to use the geometric mean than the

arithmetic mean of the AHP parameters' weights.

We re-implemented the effective parameters' weights

using the geometric mean of all weights across all

respondents.




There are two approaches [Aragon, et al (2012)]

for solving the effective parameters:

(1) EIW: Effective Individual Weights

computes the individual parameters' weights and

get their geometric mean

(2) WEPM: Weights of the Effective Preference Matrix

get the geometric mean of all the preference matrices

and compute the parameters' weights.


We implemented both approaches in combination

with the 3 AHP algorithms for comparison and validation.


Finally, we will use the following result

(using the Weights of the Effective Preference Matrix

of the Mean of Normalized Values AHP Algorithm)


AHP allows hierarchies/subcategories

Phase III for gathering the sub-categorical weights or

adjustment factors

AHP | III. SUBCATEGORY WEIGHTS


Geometric Mean of all survey data


(Context is Per Estero)

Computed Unit Market Value =

Average Market Value * (

Land-Use * (Commercial|Industrial|Residential…)

+ Accessibility *(Proximity to POIs and Access to Roads)

+ Lot Area * (Preferred|Not-Preferred)

+ Lot Shape * (Quadrilateral|NonQuadrilateral)

+ Neighborhood Classification * (Formal|Informal)

)

AHP | FINAL PARAMS AND WEIGHTS

(Context is Per Estero)

Computed Unit Market Value =

Average Market Value * (

0.4287 * (1.5148l|1.1308|1.1288|1.0080|1.0000)

+ 0.2809 *(0..1)

+ 0.1119 * (1.5599|0.3338)

+ 0.0988 * (1.3831|0.5997)

+ 0.0797 * (1.4082|0.5696)

)

AHP | FINAL PARAMS AND WEIGHTS

AHP | GIS

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Key_aspects_of_GIS/00v20000000r000000/

AHP | ArcPy

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Working_with_geometry_in_Python/002z0000001s000000/

AHP | ArcPy

AHP | ArcPy

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Working_with_geometry_in_Python/002z0000001s000000/

AHP | ArcPy

AHP | ArcPy

AHP | ArcPy

AHP | ArcPy

AHP | ArcPy

AHP | ArcPy

AHP | ArcPy

AHP | ArcPy

AHP | ArcPy

AHP | ArcPy

AHP | ArcPy

AHP | VALIDATION

AHP | WELCH’S TEST

AHP | MARKET VALUE MAP



Pure Python Pipeline:

SPSS <= RPy2 or Pandas (Python Data Analysis Library)

ArcGIS (ArcPy) <= QGIS (PyQGIS)

RECOMMENDATIONS

http://www.bytemining.com/2010/10/accessing-r-from-python-using-rpy2/

http://pandas.pydata.org/

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Key_aspects_of_GIS/00v20000000r000000/

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/A_quick_tour_of_Python/002z00000023000000/

http://www.qgis.org/en/site/

http://www.qgis.org/en/docs/pyqgis_developer_cookbook/index.html




Market Value Map

GENERAL PROCESS FLOW

ArcPy

http://ithelp.port.ac.uk/images/SPSS-logo-32F23C8B51-seeklogo.png

http://www.lic.wisc.edu/training/Images/arcgis.gif

http://www.logilab.org/




Market Value Map

RECOMMENDATIONS

PyQGIS

http://pandas.pydata.org/

http://rpy.sourceforge.net/rpy2/doc-dev/html/index.html

http://trac.osgeo.org/qgis/chrome/site/qgis-icon.png

RECOMMENDATIONS | BOOKS

http://locatepress.com/

This comprehensive article demonstrates the tight integration

of Python’s data analysis and geospatial libraries:

IPython

Pandas

Numpy

Matplotlib

Basemap

Shapely

Fiona

Descartes

PySAL

RECOMMENDATIONS | MASHUP

http://sensitivecities.com/so-youd-like-to-make-a-map-using-python-EN.html

There are two types of expertise.

One is the type you already know – content expertise,

immersing yourself deeper and deeper in a subject,

practicing for 10,000 hours and all of that.

But I think there’s a connection expertise too.

That comes from going horizontal rather than vertical.

It’s about knowing a little about a lot,

and finding wisdom in how things connect in new and different ways.

MICHAEL STANIER

http://www.speakers.ca/wp-content/uploads/2012/12/Michael-Bungay-Stanier_Feb2-760x427.jpg

Python could be a valuable tool for expanding your knowledge

vertically, as well as horizontally. And, it’s a must have tool for

connectionist experts.

END NOTE

http://fc01.deviantart.net/fs25/i/2009/022/1/a/inject_knowledge_question_mark_by_CHIN2OFF.jpg

Aragon, T., et al (2012). Deriving Criteria Weights for Health Decision Making: A Brief

Tutorial, http://www.academia.edu.

Forman, E. & Selly, M. (2001). Decision By Objectives: How to Convince Others That

You Are Right. World Scientific Publishing Co. Pte. Ltd. Singapore.

Griffiths, D. (2009). Head First Statistics. O’Reilly Media, Inc., 1005 Gravenstein Highway

North, Sebastopol, CA 95472. USA.

Ishizaka, A. & Lusti, M. (2006). How to Derive Priorities in AHP: A Comparative Study.

Central European Journal of Operations Research, Vol. 14-4, pp. 387-400.

Lamata, M. & Pelaez, J. (2002). A Method for Improving the Consistency of Judgements.

International Journal of Uncertainty, Fuzziness, and Knowledge-Based

Systems. Vol. 10, No.6, pp. 677-686. World Scientific Publishing Company.

Pelaez, J. & Lamata, M. (2002). A New Measure of Consistency for Positive Reciprocal

Matrices. Computers and Mathematics with Applications, 46 (8), pp. 1839-1849.

Pornasdoro, K. & Redo, R. S. (2011). GIS-Assisted Valuation Using Analytic Hierarchy Process

and Goal Programming: Case Study of the UP Diliman Informal Settlement Areas

(Undergraduate Thesis).

Uysal, M. P. (2010). Analytic Hierarchy Process Approach to Decisions on Instructional

Software. 4th International Computer & Instructional Technologies Symposium,

Selçuk University, Konya, Turkey, pp. 1035-1040.

REFERENCES

Thank You!

Technology

PyCon PH 2014 - GeoComputation