GEOBIA –Evolving Beyond Segmentation

GEOBIA –Evolving Beyond Segmentation

GEOBIA 2014, Thessaloniki, GreeceKeynote Presentation – May 24, 2014

Geoffrey J. Hay (PhD)B. Hemachandran, M. Rahman, B. Karim, I. Couloigner, R. Kamberov

Foothills Facility for Remote Sensing and GIScience

Department of Geography, University of Calgary, Alberta, Canada, Email: gjhay@ucalgary.ca

© 2014  G.J.Hay

Briefly describe GEOBIA evolution through changes to its definition.

Provide a new evolved definition of GEOBIA Provide 2 challenges to the GEOBIA Community Describe an award winning GEOBIA GeoWeb project  Briefly describe 4 NEW GEOBIA technologies based on existing GIS Vector Objects

2

Outline

© 2014  G.J.Hay

How long have you been ‘involved’ in GEOBIA research? 

20+ yrs? 10+ yrs? 5+ yrs? 3+ yrs? 1+ yrs?

We are a young discipline

3

What is GEOBIA to You?

© 2014  G.J.Hay 4

Brief Evolution of GEOBIA Definitions

GEOBIA

Barrysworld.biz

OBIA?

© 2014  G.J.Hay 5

OBIA 2006

© 2014  G.J.Hay

Wikipedia “OBIA”: An obia (also spelt obeah) 

is a monster in West African Folklore

What is OBIA? 

Google “OBIA”: Offshore Biologically

Important Area Ontario Brain Injury

Association Oregon Building Industry

Association

Source: http://www.cinemastrikesback.com/news/daily/king-kong-1000-4.jpg

6

© 2014  G.J.Hay

Lacking a formal definition – we made one up…

“…OBIA is a sub‐discipline of GIScience devoted topartitioning remote sensing (RS) imagery into meaningfulimage‐objects, and assessing their characteristics throughspatial, spectral and temporal scale. At its most fundamentallevel, OBIA requires image segmentation, attribution,classification and the ability to query and link individualobjects (a.k.a. segments) in space and time…”

Hay and Castilla (OBIA, 2006)

7

OBIA Definition #1

© 2014  G.J.Hay

GEO‐OBIA Tenets  Earth centric: its data sources originate from 

the surface of this planet Multi‐source capable: its methods provide 

for the integration of many different digital data types/sources

Geo‐object‐based: geo‐object segmentationis a pre‐requisite

Multi‐scale: a scene is composed of image‐objects of different size, shape and spatial arrangement

Contextual: it has the ability or integrate‘surrounding’ information and attributes.

Adaptive: it allows for the inclusion of human semantics and hierarchical networks to generate new Geographic Information

© 2014  G.J.Hay

“…GEOBIA is a sub‐discipline of GIScience devoted devoted todeveloping automated methods to partition remote sensingimagery into meaningful image‐objects, and assessing theircharacteristics through spatial, spectral and temporal scales,so as to generate new geographic information in GIS‐ready

format…”

Geographic Object‐Based Image Analysis (GEOBIA): A new name for a new discipline (Hay and Castilla, 2008)

9

GEOBIA Definition #2

© 2014  G.J.Hay 10

GEOBIA 2008

© 2014  G.J.Hay

Innovation+ transdisciplinary,

new product/solution markets

Identification+ automated object detection

+ feature extraction

Integration+ web-shared semantic models,

Geo-database architectures, bridging of RS+GIS

Interpretation+ expert rules, + semantic networks + model building

Images + multiscale, multisource

Intelligence Acquisition+ geo-content in context

11

© 2014  G.J.Hay

“…GEOBIA is a sub‐discipline of GIScience devoted devoted todeveloping automated methods to partition remote sensingimagery into meaningful image‐objects, and assessing theircharacteristics through spatial, spectral and temporal scales.Its primary objective is the generation of geographicinformation (in GIS‐ready format) from which new geo‐intelligence can be obtained. Here, geo‐intelligence isdefined as geospatial content in context… ”

Forward ‐ 2010 PE&RS GEOBIA Special Issue, Hay and Blaschke (2010) 

12

GEOBIA Definition #3

© 2014  G.J.Hay 13

GEOBIA 2010

© 2014  G.J.Hay

Evolving Community Needs

Processing/Methods: BIG (LOTs of) Data Capable H‐Resolution Capable Easily implemented Automated  Repeatable, Transferable, 

Shared Software‐as‐a‐service…

Results: Diverse Geo‐Information 

Products Validated (Error Analysis) Meaningful for Multiple 

Users Web Accessible (VGI 

inputs) Machine Usable – OGC 

Standards

14

© 2014  G.J.Hay 15

GEOBIA 2012

© 2014  G.J.Hay

Is GEOBIA a sub‐discipline of Geoinformatics or GIScience?

Geoinformatics has been described as "the science and technology dealing with the structure and character of spatial information, its capture, its classification and qualification, its storage, processing, portrayal and dissemination, including the infrastructure necessary to secure optimal use of this information”(http://en.wikipedia.org/wiki/Geoinformatics)

GIScience is “the science behind the systems,” it’s concerned with a set of fundamental questions raised by GIS and allied technologies (Goodchild, 1992).

16

Round‐Table:GeoInformatics vs. GIScience

© 2014  G.J.Hay 17

Branches of Geoinformatics

Cartography

Source: http://en.wikipedia.org/wiki/Geoinformatics

Geodesy GIS

GNSS Photogrammetry Remote Sensing

Spatial Analysis

Web Mapping

© 2014  G.J.Hay

“… GEOBIA is a sub‐discipline of GeoInformatics focused on(i) developing automated methods to extract meaningfulimage‐object attributes from remote sensing imagery, (ii)assessing their multi‐scale characteristics through time andspace, (iii) and generating new geo‐intelligence (i.e.,geospatial content ‘in context’) from related multi‐sourcedatasets….”

Hay, (GEOBIA, 2014)

18

GEOBIA Definition #4

© 2014  G.J.Hay

Extract vs. Segment”… extract meaningful image‐object attributes from remote sensing imagery…”

Segmentation is an ill‐posed problem, in the sense it has no unique solution, e.g., 

Even human photo‐interpreters will not delineate exactly the same things…

Why segment if image‐objects are well represented by existing GIS vectors?

Dubé. P and G.J.Hay, 1998

© 2014  G.J.Hay

Geoffrey M. Smith and R. Daniel Morton, 2010. Real World Objects in GEOBIA through the Exploitation of Existing Digital Cartography and Image Segmentation, PE&RS Vol. 76, No. 2, Feb, pp. 163–171 

20

Use Existing GIS Vector Image‐Objects

Cadastral vectors overlaid 1.0 m ortho-air photography

Cadastral vectors overlaid 25 m TM imagery

© 2014  G.J.Hay

Evolving Community NeedsInput: Earth‐Centric Multisource Capable Geo‐object‐based Multi‐scale Contextual Adaptive

Processing/Methods: BIG (LOTs of) Data Capable H‐Resolution Capable Easily implemented

21

Automated  Repeatable, Transferable, Shared Software‐as‐a‐service…

Results: Diverse Geo‐Information Products Validated (Error Analysis) Meaningful for Multiple Users Web Accessible (VGI inputs) Machine Usable – OGC Standards

© 2014  G.J.Hay

Based on increasingly sophisticated user needs, and the fact that segmentation has no unique solution, we advocate …that GEOBIA segmentation technology (though not perfect) has sufficiently evolved, and challenge the GEOBIA community to:

1. Rediscover the wealth of existing high quality GIS vector data, and where appropriate use these as segmented image‐objects;

2. Redirect research towards generating geo‐web accessible ‘geo‐intelligence’. 

22

GEOBIA Community Challenge

© 2014  G.J.Hay

HEAT (Heat Energy Assessment Technologies) is a Free GeoWeb service,designed to help residents (i) improve their Home Energy Efficiency,(ii) save their money, and (iii) reduce their green-house-gas (GHG)emissions by discovering their HEAT Score and visualizing the amountand location of waste heat leaving their homes and communities, as easilyas clicking on their house in Google Maps.

23

Data Sets Thermal Imagery:

– 44 Flight Lines (630 GB) TABI‐1800 (Thermal Airborne Broadband Imager)

– Swath width: 1800 pix, @50 cm

– Spectral region: 3.7‐4.8 µm– Thermal res’: 0.05 C– Ability to collect up to 175 km2

per hour at 1.0 m 

City Data:– The City of Calgary GIS

building layers, RGB‐NIR ortho‐photography

– MLS: Roof materials

GEOBIA:– Roof Feature Detection

24(Figure) 15 cm TABI 1800 image showing community, houses, vehicles in driveway, roof structure and tree wind shadow (April, 2011).

© 2014  G.J.Hay 25(Figure) 15 cm TABI 1800 image showing community, houses, vehicles in driveway, roof structure and tree wind shadow (April, 2011).

A B

CD

ww

w.s

aveh

eat.c

o

ww

w.s

aveh

eat.c

o

27

ww

w.s

aveh

eat.c

o

ww

w.s

aveh

eat.c

o

ww

w.s

aveh

eat.c

o

ww

w.s

aveh

eat.c

o

© 2014  G.J.Hay

Waste HEAT Maps – Call to Action

32

© 2014  G.J.Hay 33

Community EngagementHome owners, 

rentersConstruction Companies

Real‐Estate agents

Green Service Providers

Community and City SaveHEAT Competitions.

www.andrewhuffer.com.au

© 2014  G.J.Hay

Choice of Green Energy Service Providers

Green EnergyWind, solar, geothermal

Energy Efficient Products windows, doors, roofing

HVAC Systems

Solarphotovoltaic, hot water

Carbon Offset Purchases

Green roofsGreen Construction18

© 2014  G.J.Hay

Near Future ‐MyHEAT City ProgramI. Scale‐up and apply in 

cities with populations greater than one million inhabitants; 

II. Implement over time to establish monitoring programs;

III. Initiate Save‐heat competitions between and within different communities and cities. 

IV. Develop solutions for the industrial, commercial and municipal building sectors

35

© 2014  G.J.Hay

Near Future ‐MyHEAT City ProgramI. Scale‐up and apply in 

cities with populations greater than one million inhabitants; 

II. Implement over time to establish monitoring programs;

III. Initiate Save‐heat competitions between and within different communities and cities. 

IV. Develop solutions for the industrial, commercial and municipal building sectors

36

© 2014  G.J.HayUrban Waste Heat Plumes, G. J. Hay Calgary Feb, 2011

We estimate Total Calgary Savings (Natural Gas) of $33,564,386 and a reduction of 198,216T of CO2e per/yr

20

© 2014  G.J.Hay* Alex Steffen – SXSW Eco Conference Keynote (Oct 05, 2011)

“... The great killer app of energy use: … letting people know how they’re doing

relative to others…

Comparison will change behaviours very quickly, because nobody wants to

be the outlying energy hog...”*

38

Our Mission is to show 'what urban energy efficiency looks like', 'where it is located', 'what it costs' and 

'what to do about it'. 

HEAT Mission

39

We believe that if people could see the waste heat they generate and if they knew how much it cost (financially and to the environment), that they would want to take action. 

We want to show them how.

HEAT Mission

40

© 2014  G.J.Hay

Won from over 400 contestants world‐wide;

25 Media Interviews: Provincial, National, International;

Over 10 days received 40,343 visits, 110,962 page‐views from 1017 cities in 97 countries in 38 different languages.

Interest from 6 core sectors: (i) Energy, (ii) Banks and Investment, (iii) Residential Construction, (iv) Real‐Estate, (v) Information Technology and (vi) Governments/Municipalities

MIT – Climate CoLab2013 Grand Prize Winner

41

42

© 2014  G.J.Hay

HEAT Maps/Metrics ‐ 44 flight‐lines of H‐Res TABI 1800 airborne imagery (630GB at 50cm and 0.05 deg. C):

1. TURN ‐ Thermal Urban Road Normalization

2. RRN ‐ Relative Radiometric Normalization (RRN)

3. OBM ‐ Geo‐Object Based Mosaicing (OBM) 

4. OBGC ‐ Geo‐Object‐Based Geometric Correction (OBGC) 

43

4 NEW GIS‐Based Geo‐Object Technologies

© 2014  G.J.Hay

Thermal Urban Road Normalization (TURN) ‐reduces the microclimatic variability within thermal flight‐lines based on temperatures extracted from (GIS buffered) road‐objects(and interpolated from 10, 20,50 100 m samples for 825sq km).

The first instance of discrete geo‐objects (roads) being used to create continuous surfaces.

44

TURN

© 2014  G.J.Hay

Relative Radiometric Normalization (RNN) ‐(automatically) samplesimage‐object groups in the overlap between flight‐lines to create non‐linear models that mitigate the between flight‐line radiometric variability

So it appears that all flight‐lines were acquired at the same time

45

RRN

Rahman & Hay, 2014. A comparison of four relative radiometric normalization(RRN) techniques for mosaicing H-res multi-temporal thermal infrared (TIR)flight-lines of a complex urban scene.” In Review, RSE-D-14-00440.

© 2014  G.J.Hay

OBM

House

House

House

House

House

House

Traditional Mosaic

OBM

Geo‐Object Based Mosaicing (OBM) – uses GIS roof polygons to automatically mosaic around 14,209 individual roofs, rather than arbitrarily bisecting them along mosaic join‐lines

Resulting in improvedthermal based maps and metrics, i.e., Hot‐spot detection

46

© 2014  G.J.Hay

Object‐Based Geometric Correction (OBGC) – Automatically and accurately ties different high‐resolution (i.e., 25cm – 1m) images of the same location – but acquired with different spatial, spectral and temporal resolutions ‐ to 330,000+ pre‐defined GIS house‐objects.

This effectively allows GIS polygons to automatically be used as masks to extract the image information beneath them

47

(OBGC) 

© 2014  G.J.Hay 48

(OBGC) 

© 2014  G.J.Hay

Shifting 29,000+ polygons in one of 9 cardinal directions

49

OBGCObject‐Based Geometric Correction

© 2014  G.J.Hay

Shifting 29,000+ polygons a distance of 1 – 4+ pixels

50

OBGCObject‐Based Geometric Correction

© 2014  G.J.Hay

The needs of the GEOBIA community are evolving Our definitionmust also evolve to guide these changes Segmentation is problematic  Propose 2 Challenges to the GEOBIA Community:

1. Rediscover the wealth of existing high quality GIS vector data, and where appropriate use these as segmented image‐objects;

2. Redirect research towards generating geo‐web accessible ‘geo‐intelligence’

The HEAT project is an award winning GEOBIA GeoWeb project that incorporates 4 NEW GEOBIA technologies based on existing GIS Vector Objects to generate new Geointelligence (geospatial content in context).

51

Conclusion

© 2014  G.J.Hay 52

© 2014  G.J.Hay

HEAT TEAM Members (Current): (PI) Geoffrey J. Hay (PhD,Geography, ISEEE Fellow) Bharani Hemachandran (MSc) Mustafiz Rahman (PhD Candidate, Geog) Bilal Karim (MSc Candidate, Geog) Rustam Kamberov (PhD Candidate, Geog) Isabelle Couloigner, (Research Associate, Fr.Ing, PhD) Wen Cao, (Research Associate, MSc ComSci) Tak S. Fung (PhD, Math ‐ Statistics) Joseph L. Arvai (PhD, ISEEE ‐ Decision Support) Tang Lee (PhD, EVDS ‐ Building Envelope/Efficiency) Joule Bergerson (PhD, ISEEE – Life Cycle Analysis/Modeling)

Acknowledgements

53

© 2014  G.J.Hay

GEOBIA is much more than segmentation The needs of the GEOBIA community are evolving Our definitionmust also evolve to guide these changes Segmentation is problematic  We Challenge the GEOBIA Community to

1. Rediscover the wealth of existing high quality GIS vector data, and where appropriate use these as segmented image‐objects;

2. Redirect research towards generating geo‐web accessible ‘geo‐intelligence’

The HEAT project is an award winning GEOBIA GeoWeb project that incorporates 4 NEW Geo‐Object Based technologies based on existing GIS Vector Objects

54

Conclusion