KEO Final presentation - Esrin 11/12/2008 1 Knowledge-centred Earth Observation Final presentation...

KEO Final presentation - Esrin 11/12/2008 1

Knowledge-centred Earth Observation

Final presentation

acs-keo@acsys.it

KEO Final presentation - Esrin 11/12/2008 2

ACS is a private Italian SW engineering company specialised in satellite remote sensing:

• Founded in 1979

• Headquarters in Rome

• Research Centre in Matera (since 2001)

• 110 Employees (90% graduates, 11 PhDs)

• Quality System Certification ISO 9001 (since ‘99)

• Certified balance sheet since 1999

About ACS

KEO Final presentation - Esrin 11/12/2008 3

• Integrated Systems

• Software solutions

• Services and Consultancies

In the fields of:

- Earth Observation Satellites Ground Segments

- Environment Monitoring & Management

Applications

- Advanced & Immersive data visualisationTechnology transfer Medical Applications

ACS proposes

KEO Final presentation - Esrin 11/12/2008 4

Ground Segment Technologies

• Payload Ground Segments• Processors (SAR and Optical)• Data Mining • Mission Planning Tools• User Services

ACS Capabilities

KEO Final presentation - Esrin 11/12/2008 5

EO Applications

• Vegetation Monitoring

• Detection of Marine Pollution

• Volcano Eruption Monitoring

• Environmental Risk Monitoring

• Cartography and DEM/DTM

• Desertification Monitoring

• Urban Expansion

• Applied Interferometry

ACS Capabilities

KEO Final presentation - Esrin 11/12/2008 6

ACS Systems in the World

KEO Final presentation - Esrin 11/12/2008 7

• Knowledge-based Information Mining (KIM) tool

– Interactive image collection analysis using Probabilistic Information Mining

– Cooperation of ACS, DLR, ETHZ

– Technology development started in 2001 (ESA TRP) and improved over time

• Knowledge-centred Earth Observation (KEO)

– KIM + rich, graphic, distributed environment for automated Information Extraction

– Cooperation of ACS, DLR, CNES, GTD

– Started in 2005 (ESA TRP)

• Image Information Mining - Time Series (IIM-TS)

– KEO modules for co-registration and Time Series handling

– Cooperation of ACS, CNES, DLR, IGUASSU, JRC, MEEO, SARMAP, UNIPV, UNITN, VTT

– Started in 2007 (ESA TRP)

ACS & IIM

KEO Final presentation - Esrin 11/12/2008 8

The System

KEO Final presentation - Esrin 11/12/2008 9

What is KEO ?

• Distributed Component-based Processing Environment

• Graphically build of Processing Components• Image Information Mining (IIM) Processing with

Interactive Training• Processing results available through OGC

services (WMS, WFS, …)• Processing graphs and/or processing results are

easy to publish in SSE

KEO Final presentation - Esrin 11/12/2008 10

KEO SW architecture

KEO Final presentation - Esrin 11/12/2008 11

KEO HW Architecture

KEO Final presentation - Esrin 11/12/2008 12

FP HW/SW architecture

KEO Final presentation - Esrin 11/12/2008 13

KEO Users

• Domain Expert– Not an expert in remote

sensing– Needs information to make

decision

• Remote Sensing Expert– A scientist with signal & image

processing know-how– Able to build chains of

processing components

KEO Final presentation - Esrin 11/12/2008 14

Live Demos

KEO Final presentation - Esrin 11/12/2008 15

KEO Users Interfaces

• SSE [Domain Expert]• http://keo.esrin.esa.int • KAOS [Domain & Remote Sensing Experts]

– Server Monitor– Image Browser– Image Info Viewer– Product Explorer– World Wind Viewer– FEP Designer

KEO Final presentation - Esrin 11/12/2008 16

Live Demo Glossary

• FEP (Feature Extraction Processor)

• FEP Engine

• FEP Actuator

• Data Flow

• Module [CLI, JAVA, WS]

• Ports

• Connection (Semantic Check)

KEO Final presentation - Esrin 11/12/2008 17

Live Demo NDVI 1

• Problem– Vegetation Detection using NDVI

• Solution– Search in KEO a NDVI modules

• Action– Run the found FEP

KEO Final presentation - Esrin 11/12/2008 18

Live Demo NDVI 2

• Problem– NDVI isn’t good enough

• Solution– Search in KEO for different NDVI modules

• Action– Create a new FEP with the found CLI module

and run it

KEO Final presentation - Esrin 11/12/2008 19

Live Demo NDVI classification

• Problem– Classify the NDVI image to find an easy way

to detect a particular type of vegetation

• Solution– Use an available classification algorithm

• Action– Create a new CLI with the classifier and

integrate it within our FEP

KEO Final presentation - Esrin 11/12/2008 20

Knowledge-based Information Mining

KIM

Live Demo KIM subsystem

KEO Final presentation - Esrin 11/12/2008 21

Live Demo Motivation

• KEO/FEP: a tool for large-scale algorithmic processing– For well-specified phenomena of interest– With well-known solutions

• KEO/KIM: a tool for Exploratory Analysis– For unknown / hard-to-specify phenomena– Via examples [using color, shape, multi-scale

texture for optical, radar sensors]– Interactive KEO processor definition

KEO Final presentation - Esrin 11/12/2008 22

Live Demo Image Browser

• An image archive client:– browse by quicklook

visualization– search by attributes

(e.g. date, footprint...)

– search by content!

KEO Final presentation - Esrin 11/12/2008 23

Live Demo Label Designer

• A rich interaction environment

• Iterative process:– YES/NO examples

via mouse clicks– interface reacts:

red interest areas

• Zoom levels, undo, real-time enhancement...

YES

NO

KEO Final presentation - Esrin 11/12/2008 24

Live Demo Label Designer

• Fast, iterative supervised training across archive

• Can be re-used, refined, extended, managed

• Associated to interactively user-defined textual tags

• Seamless data exchange with archive

KEO Final presentation - Esrin 11/12/2008 25

Live Demo Thematic Maps

• User-defined interest rapid mapping

• Automatic production on-the-fly for the entire archive

• Select + rank relevant images by content

• Can be published on SSE or WCS/WMS

KEO Final presentation - Esrin 11/12/2008 26

Live Demo Product Explorer

• Expert exploratory analysis

• Define new product format combining – Thematic maps– Primitive signals

for further analysis / other users

KEO Final presentation - Esrin 11/12/2008 27

KIM: image search engine

• Application-independent descriptors

• Content simplifier

• Interactivesub-space selection + tagging by labels

Google, Yahoo

• Crawler - data collection / analysis

• Content indexer

• Interactive front end /ranking

“ingestion”

Evolution

KEO Final presentation - Esrin 11/12/2008 28

System Architecture

KEO Final presentation - Esrin 11/12/2008 29

Main componets

• Ingestion subsystem

• FEP subsystem

• KARISMA application

• Reference DataSet

• Ontology subsystem

• OGC services

• KAOS client application

KEO Final presentation - Esrin 11/12/2008 30

Ingestion subsystem

In charge of ingesting products extracted and/or computing:

• information for KIM processing

• features and KIM data

• feature labels and labels maps

• co-registered products

KEO Final presentation - Esrin 11/12/2008 31

FEP subsystem

A data-flow engine in charge of extracting features using Feature Extraction Processors made of:

• Java-coded modules

• CLI programs

• external Web Services

KEO Final presentation - Esrin 11/12/2008 32

FEP subsystem

Made of:• Designer, used to graphically

draw a processing graph• Engine, the executor and

coordinator of the processing graph

• Actuator, the executor of the platform-dependent modules

KEO Final presentation - Esrin 11/12/2008 33

FEP Engine

KEO Final presentation - Esrin 11/12/2008 34

FEP Actuator

KEO Final presentation - Esrin 11/12/2008 35

KARISMA

• The KEO back end application for KIM– Provides access to image archive– Provides access to KIM data– Implements most of the KIM engine– Manages users, projects, …

• J2EE application and some W/S

• Deployed on a glassfish application server

KEO Final presentation - Esrin 11/12/2008 36

KAOS

• KEO client application– KIM client

[KARISMA]– FEP client

[KAFE]– KEO system

management

KEO Final presentation - Esrin 11/12/2008 37

AIR web Service

• Automatic Image Registration– First level of integration of GDAL, but still semi-automatic– Configuration files added for processing different image pairs– Command Line Interface simplification and logfiles centralization

• Deformation Model Management System– XML files based on OGC standards– C++ library for handling a Deformation Model– Test and visualization tools– Some anomalies still to be corrected

• Web Service (AIR-WS)– AIR-WS Java application developed on top of AIR Python scripts– Test and visualization tools

KEO Final presentation - Esrin 11/12/2008 38

Ontology subsystem

Free text search inside KEO by KAOS client

KEO Final presentation - Esrin 11/12/2008 39

Reference Data Set

• Set of reference data used to test and validate FEPs

• Collection of georeferenced Images and any other relevant data (text, pdf, movie, URLs, …)

• Based on GeoNetwork

KEO Final presentation - Esrin 11/12/2008 40

GeoNetwork Features

Users• Search in multiple

catalogues through one website

• Access data from source site

• File download• Data visualization through

web map viewer

Data Providers• Standardized process to

document and publish• Online metadata editor• Different metadata

standards (ISO 19115, ISO 19139, FGDC, Doublin Core)

• Different sharing / security levels

• File upload

KEO Final presentation - Esrin 11/12/2008 41

GeoNetwork Features

GeoNetwork Technology• Java language (platform

independent)• Servlet environment (Tomcat,

Jetty, ...)• XML data representation

(JDOM)• XSL templates for

presentation (XALAN)• JDBC for database access

(runs on any compliant DBMS)

• XML Schema for metadata editing

Catalogue interoperability• Full OGC Catalog v2.0 support• ISO profile

(Reference OGC implementatio)

• Open Archive Initiative support (OAI-MPH)

• RSS and GeoRSS newsfeeds• OpenSearch with GEO

extension• Metadata harvesting &

synchronization

KEO Final presentation - Esrin 11/12/2008 42

OGC services

• WMS and WFS for data extracted by PIM Batch processors

• Based on UMN MapServer

• WMS and WFS exposed to SSE

KEO Final presentation - Esrin 11/12/2008 43

System Evolutions

KEO Final presentation - Esrin 11/12/2008 44

• Technology and Research continues with IIMTS project aimed at extending KEO and KIM capabilities to Image Time Series with the purpose of:

– Identifying and locating changes– Identifying and locating patterns– Tracking and following movement of recurring features

which could represent potentially usefulinformation for specific applications:

– Urban areas monitoring– Agricultural monitoring– Emergency and Security

Further Evolutions in IIM

KEO Final presentation - Esrin 11/12/2008 45

• KEO provides a distributed environment where Feature Extraction Algorithms (including Time Series) can be plugged and used as simple CLI modules (in any programming language).

• Any KEO registered user can deploy his own modules and make these modules usable by others thus optimising development efforts

• By design CLI modules run on processing nodes that can even be physically located on remote sites thus providing an effective solution for a distributed exploitation and processing environment

……………….. And in future ?

Conclusions and Future

KEO Final presentation - Esrin 11/12/2008 46

• Classification of Optical Images (including Very High resolution)

• Tools for Oil Spill and Ship Detection from SAR images

• Binary and Multiclass Change Detection tools from Optical and SAR Images

• Shape Change Detection Tools to point out geometric differences between regions from previous classifications

• Trend Analysis in Temporal Time Series also as an extension of the KIM concept.

From May 2009 KEO will include Processing Components (developed within the Image Information Mining – Time Series project) in support to:

Conclusions and Future

KEO Final presentation - Esrin 11/12/2008 47

• A KEO system will be installed in Romania in the frame of the ESA-ROSA (Romanian Space Agency) agreement.

• ACS continues to study and prototype a second generation of KIM Systems

• A discussion with ESA is undergoing to define a working plan for KIM/KEO enhancements

• An assessment of what should come first in terms of evolutions is needed, so feedback from users would be highly appreciated ………..

………….. What will happen in ‘09 ?

Conclusions and Future

KEO Final presentation - Esrin 11/12/2008 48

• People interested in becoming registered KEO users are invited to contact ESA!http://keo.esrin.esa.int

• For any question please contact:

– ESA for KEO: • Michele Iapaolo michele.iapaolo@esa.int

– ESA for RDSs:• Andrea Della Vecchia andrea.della.vecchia@esa.int

– ACS: • KEO Team acs-keo@acsys.it

Thank you for your Attention

Conclusions and Future

KEO Final presentation - Esrin 11/12/2008 49

Questions?