Center forSubsurface Sensing & Imaging Systems

Center forSubsurface Sensing & Imaging Systems

Overview of Image and Data Information

Management in CenSSIS

David Kaeli Northeastern University

Boston, MAkaeli@ece.neu.edu

R1

R2

Overview of the Strategic Research PlanOverview of the Strategic Research Plan

FundamentalScienceFundamentalScience

ValidatingTestBEDsValidatingTestBEDs

L1L1

L2L2

L3L3

R3Image and Data

InformationManagement

S1 S4 S5S3S2

Bio-Med Enviro-Civil

R3 Research Thrust OverviewR3 Research Thrust Overview

Utilize enabling hardware and software technologies to address CenSSIS barriers

Pursue research in enabling technologies Develop a common set of tools and techniques to

address SSI problems: Hardware parallelization and acceleration Software toolboxes Image database management and tools

Utilize enabling hardware and software technologies to address CenSSIS barriers

Pursue research in enabling technologies Develop a common set of tools and techniques to

address SSI problems: Hardware parallelization and acceleration Software toolboxes Image database management and tools

Toolboxes

CenSSIS Middleware Tools CenSSIS Middleware Tools

Parallelization of MATLAB, C/C++ and Fortran codes using Message Passing Interface (MPI) – a software pathway to exploiting GRID-level resources

Utilizing MPI-2 to address barriers in I/O performance Building on existing Grid Middleware such as Globus

Toolkit, MPICH-G2 and GridPort Presently illustrating the impact of the GRID on system

level projects (tomosynthesis reconstruction)

Parallelization of MATLAB, C/C++ and Fortran codes using Message Passing Interface (MPI) – a software pathway to exploiting GRID-level resources

Utilizing MPI-2 to address barriers in I/O performance Building on existing Grid Middleware such as Globus

Toolkit, MPICH-G2 and GridPort Presently illustrating the impact of the GRID on system

level projects (tomosynthesis reconstruction)

MATLAB

C/C++

Fortran

Parallelization

MPI

MPICH-G2

UPC

Impact on CenSSIS ApplicationsImpact on CenSSIS Applications

Reduced the runtime of a single-body Steepest Descent Fast Multipole Method (SDFMM) application by 74% on a 32-node Beowulf cluster• Hot-path parallelization• Data restructuring

Reduced the runtime of a Monte Carloscattered light simulation by 98% on a 16-node Silicon Graphics Origin 2000• Matlab-to-C compliation• Hot-path parallelization

• Obtained superlinear speedup of Ellipsoid Algorithm run on a 16-node IBM SP2 • Matlab-to-C compliation• Hot-path parallelization

Reduced the runtime of a single-body Steepest Descent Fast Multipole Method (SDFMM) application by 74% on a 32-node Beowulf cluster• Hot-path parallelization• Data restructuring

Reduced the runtime of a Monte Carloscattered light simulation by 98% on a 16-node Silicon Graphics Origin 2000• Matlab-to-C compliation• Hot-path parallelization

• Obtained superlinear speedup of Ellipsoid Algorithm run on a 16-node IBM SP2 • Matlab-to-C compliation• Hot-path parallelization

Soil

Air

Mine

Scattered Light Simulation Speedup

1

10

100

1000

10000

100000

Ru

nti

me

in s

ec

on

ds

Original

Matlab-to-C

Hot pathparallelization

Ellipsoid Algorithm Speedup(versus serial C version)

05

101520

1 2 4 8 16

Number of Nodes

Sp

ee

du

p

64-vector 256-vector1024-vector linear speedup

Tomographic mammographyTomographic mammography

3D image reconstruction from x-ray projections Used to detect and diagnose breast cancer Based on well developed mammography techniques Exposes tissue structure using multiple projections from different angles

Advantages Accuracy: provides at least as much useful information than x-ray film Flexibility: digital image manipulation, digital storage Structural information: using layered images Safe: low-dose x-ray Lower cost: compared to MRI

3D image reconstruction from x-ray projections Used to detect and diagnose breast cancer Based on well developed mammography techniques Exposes tissue structure using multiple projections from different angles

Advantages Accuracy: provides at least as much useful information than x-ray film Flexibility: digital image manipulation, digital storage Structural information: using layered images Safe: low-dose x-ray Lower cost: compared to MRI

Image acquisition/reconstruction processImage acquisition/reconstruction process

Acquisition: 11 uniform angular samples along Y-axis X-ray projection: breast tissue density absorption radiograph Algorithm: constrained non-linear convergence and iterative process Uses a Maximum Likelihood Estimation

Acquisition: 11 uniform angular samples along Y-axis X-ray projection: breast tissue density absorption radiograph Algorithm: constrained non-linear convergence and iterative process Uses a Maximum Likelihood Estimation

detector

X-ray source

X

Z

Y

Y

Set 3D volume

Compute projections

Correct 3D volume

3D volume

Satisfied ?NoYes

Exit

Initialization

Forward

Backward

X-rayprojections

Parallelization approachesParallelization approaches

Reduce communication data Segmentation along Y-axis Using redundant computation to replace communication Segmenting along x-ray beam

Reduce communication data Segmentation along Y-axis Using redundant computation to replace communication Segmenting along x-ray beam

First approach:Non inter-communication(more computation, less communication)

Second approach:Overlap with inter-communication

Third approach:Non-overlap with inter-communication(less computation, morecommunication)

exchange dataOverlap area

Tomosynthesis AccelerationTomosynthesis Acceleration

Phantom data test results using non-overlap method on 32 CPUs

0

50

100

150

200

250

300

350

P4 cluster Hypnoscluster

Titancluster

IBM p690 SGI Altix

Platform

Tim

e (s

ec)

File IOCollectInter-commSyncBackwardForward

•Input data set: phantom 1600x2034x45

• Serial implementations runs in 2-3 hours on a P4 machine

• Platforms:

– SGI Altix system

– UIUC NCSA Titan cluster

– UIUC NCSA IBM p690

– UMich Hypnos cluster

– P4 cluster at MGH

• Number of processors: 32

Computation: SGI Altix with Itanium 2 processor outperforms the other CPUs

Currently moving this work to the GRID and the Pittsburgh Supercomputer Center

Prototype running on our GRID system at NU

Field Programmable Gate Arrays for Subsurface ImagingField Programmable Gate Arrays for Subsurface Imaging

Backprojection for Computed Tomography image reconstruction Sponsored by Mercury Computer

Finite Difference Time Domain (FDTD) in hardware Collaboration with Humanitarian Demining project

Retinal Vascular Tracing in real time Collaboration with Real-time Retinal Imaging project

Phase Unwrapping Collaboration with 3-D Fusion Microscope project

Diverse problems, similar solutions:

FPGAs are particularly well suited for accelerating image processing and image understanding algorithms

Backprojection for Computed Tomography image reconstruction Sponsored by Mercury Computer

Finite Difference Time Domain (FDTD) in hardware Collaboration with Humanitarian Demining project

Retinal Vascular Tracing in real time Collaboration with Real-time Retinal Imaging project

Phase Unwrapping Collaboration with 3-D Fusion Microscope project

Diverse problems, similar solutions:

FPGAs are particularly well suited for accelerating image processing and image understanding algorithms

Retinal Vascular Tracing: Register 2-D Image to 3-D in Real TimeRetinal Vascular Tracing: Register 2-D Image to 3-D in Real Time

FIREBIRD BOARD

HOST

Direction ofblood vessel

PCI BUS

ObjectiveTo accelerate an existing retinalvascular tracing (RVT) algorithm byimplementing computation of templateresponses in reconfigurable hardware

FPGA

BL

OC

KR

AM

DESIGN

MEMORY0

IMAGEMEMORY1

RESULTS

“Smart Camera”

Direction of blood vessel

PCI BUS

Some Recent Publications on ParallelizationSome Recent Publications on Parallelization

• “Execution-Driven Simulation of Network Storage Systems,” Y. Wang and D. Kaeli, Proceedings of the 12th ACM/IEEE International Symposium on Modeling, Analysis of Computer and Telecommunication Systems, October 2004, pp. 604-611.

• “Profile-guided File Paritioning on Beowulf Clusters,” Y. Wang and D. Kaeli, Journal o f Cluster Computing, Special Issue on Parallel I/O, to appear,

• “An Object-oriented Parallel Library,” C. Oaurrauri and D. Kaeli, International Journal of High Performance of Computing and Networking, to appear.

• “Digital Tomosynthesis Mammography using a Parallelized Maximum Likelihood Reconstruction Method,” T. Wu, R. Moore, E. Rafferty, D. Koppans, J. Zhang, W. Meleis and D. Kaeli, Medical Imaging, 5368, February 2004.

• “Mapping and characterization of applications in Heterogeneous Distributed Systems,” J. Yeckle and W. Rivera , To appear in Proceed. of the 7th World Multiconference on Systemics, Cybernetics and Informatics (SCI2003).

• “Profile-Guided I/O Partitioning,” Y. Wang and D. Kaeli, Proceedings of the 17th ACM International Symposium on Supercomputing, June 2003, pp. 252-260.

• “Source-Level Transformations to Apply I/O Data Partitioning,” Y. Wang and D. Kaeli, Proceedings of the IEEE Workshop on Storage Network Architecture and Parallel IO, Oct. 2003, pp. 12-21.

• “Execution-Driven Simulation of Network Storage Systems,” Y. Wang and D. Kaeli, Proceedings of the 12th ACM/IEEE International Symposium on Modeling, Analysis of Computer and Telecommunication Systems, October 2004, pp. 604-611.

• “Profile-guided File Paritioning on Beowulf Clusters,” Y. Wang and D. Kaeli, Journal o f Cluster Computing, Special Issue on Parallel I/O, to appear,

• “An Object-oriented Parallel Library,” C. Oaurrauri and D. Kaeli, International Journal of High Performance of Computing and Networking, to appear.

• “Digital Tomosynthesis Mammography using a Parallelized Maximum Likelihood Reconstruction Method,” T. Wu, R. Moore, E. Rafferty, D. Koppans, J. Zhang, W. Meleis and D. Kaeli, Medical Imaging, 5368, February 2004.

• “Mapping and characterization of applications in Heterogeneous Distributed Systems,” J. Yeckle and W. Rivera , To appear in Proceed. of the 7th World Multiconference on Systemics, Cybernetics and Informatics (SCI2003).

• “Profile-Guided I/O Partitioning,” Y. Wang and D. Kaeli, Proceedings of the 17th ACM International Symposium on Supercomputing, June 2003, pp. 252-260.

• “Source-Level Transformations to Apply I/O Data Partitioning,” Y. Wang and D. Kaeli, Proceedings of the IEEE Workshop on Storage Network Architecture and Parallel IO, Oct. 2003, pp. 12-21.

Held again

in 2004

CenSSIS Solutionware – UPRM/NU/RPICenSSIS Solutionware – UPRM/NU/RPI

Toolbox Development Support the development of CenSSIS Solutionware that

demonstrates our “Diverse Problems – Similar Solutions” model

Develop Toolboxes that support research and education Establish software development and testing standards for

CenSSIS

Image and Sensor Data Database Develop an web-accessible image database for CenSSIS that

enables efficient searching and querying of images, metadata and image content

Develop image feature tagging capabilities

Toolbox Development Support the development of CenSSIS Solutionware that

demonstrates our “Diverse Problems – Similar Solutions” model

Develop Toolboxes that support research and education Establish software development and testing standards for

CenSSIS

Image and Sensor Data Database Develop an web-accessible image database for CenSSIS that

enables efficient searching and querying of images, metadata and image content

Develop image feature tagging capabilities

Toolboxes

Status of the CenSSIS ToolboxesStatus of the CenSSIS Toolboxes

Hyperspectral Image Analysis Toolbox (HIAT) October 2004

Multiview Tomography Toolbox (MVT) fddlib:

January 2003 (v. 1.0) July 2003 (v. 1.1)

mvt: October 2004

Rensselaer Generalized Registration Library (RGRL) September 2004

Hyperspectral Image Analysis Toolbox (HIAT) October 2004

Multiview Tomography Toolbox (MVT) fddlib:

January 2003 (v. 1.0) July 2003 (v. 1.1)

mvt: October 2004

Rensselaer Generalized Registration Library (RGRL) September 2004

HIAT

MVT

RGRL

New toolbox: Improving the quality of radiation oncology @ MGHNew toolbox: Improving the quality of radiation oncology @ MGH

Developed a 4D (3D + including time) visualization browser tool kit

Visualize Computed Tomography (CT) images, organ outlines (wire contours) and the isodose lines (treatment dosage)

Present all this information in a user friendly interface

Developed a 4D (3D + including time) visualization browser tool kit

Visualize Computed Tomography (CT) images, organ outlines (wire contours) and the isodose lines (treatment dosage)

Present all this information in a user friendly interface

4-D Visualization of Lung Tumors4-D Visualization of Lung Tumors

Dosage

4-D Visualization

The Future for CenSSIS ToolboxesThe Future for CenSSIS Toolboxes

SCIRun

Collaboration with the University of Utah

Deliver an web-accessible database forCenSSIS that enables efficient searching and querying of images, sensor data, metadata and image content

More that 4000 metadata-rich images/datasets presently available online (> 10,000 by 2006)

Database Characteristics:

• Relational complex queries (Oracle9i)

• Data security, reliability and layered user privileges

• Efficient search and query of image content and metadata

• Content-based image tagging using XML

• Indexing algorithms (2D, 3D, and 4D)

• Explore object relational technology to handle collections

CenSSIS Image Database SystemCenSSIS Image Database System

12

34

mouse embryo

CenSSIS Image Database SystemCenSSIS Image Database System

CenSSIS Image Database SystemCenSSIS Image Database System

CenSSIS Image Database SystemCenSSIS Image Database System

Utilize Machine Learning

algorithms to improve query

view

CenSSIS Image Database SystemCenSSIS Image Database System

Provides data

description associated with initial collection,

but does not allow for further

elaboration or

annotation.

Image AnnotationImage Annotation

Provide the ability to markup image with searchable features

Enable image database to be more effectively data-mined

Provide the ability to markup image with searchable features

Enable image database to be more effectively data-mined

<xml version=“1.0” encoding=“UTF-8”>

<embryo>

<description> Embryo developmental stages</description>

<feature label=“1” xPos1=“29” yPos1=“33” xPos2=“48” yPos2=“50”> 1 cell embryo </feature>

<feature label=“2” xPos1=“50” yPos1=“28” xPos8=“70” yPos2=“40”> 2 cell embryo

</feature>

<feature label=“3” xPos1= “5” yPos1= “5” xPos2=“25 yPos2=“20”> 4 cell embryo </feature>

</embryo>

XML and JavaXML and Java

• XML (Extensible Markup Language)

• Provides maximum flexibility and portability

• Well-supported standard

• Powerful querying tools available in Oracle

• The Java2 Platform

• Cross-platform compatibility

• Standard web-browser interface

• Native XML support

Image TaggingImage Tagging

A raw image file from the CenSSIS Database

• QUERY: I want to be able to add to this image textual annotations, providing my medical team with questions about particular ROIs:

• Difficult to describe regions in an image

• Difficult to pinpoint specific features in images

• Global image metadata too coarse to facilitate low-level tagging

Image TaggingImage Tagging

Image with tags

• Metadata associated with specific areas

• Query for specific image features

The Image Tagging InterfaceThe Image Tagging Interface

Drawing Tools

View Options

Tag Options

Tags and XMLTags and XML

<feature type="Ellipse" label="4 Cell Stage"> <ellipse> <xCenter> 101 </xCenter> <yCenter> 58 </yCenter> <xRadius> 79 </xRadius> <yRadius> 46 </yRadius> </ellipse> <note> [custom XML tags go here] </note> <annotator> awilliam </annotator> </feature>

The Future Role of Image AnnotationThe Future Role of Image Annotation

Provide a vehicle for natural collaboration

• A richer set of metadata to enable more detailed queries

• Potential to perform extensive data mining on image content

• An eye toward content-based image retrieval

Tumor tracking paper recently accepted to SIGMOD 2005

The CenSSIS Image Database SystemThe CenSSIS Image Database System

Hosts the image and sensor data of CenSSIS (>500 images online) http://censsis-db1.ece.neu.edu/

Provides metadata indexed image searching Uses XML tags to allow for easy information interchange Evolved into a project-based management system, allowing

users to organize their data hierarchically Key issue: how do we develop collaboration tools that

increase the value of data stored in the database? Presently exploring how best to integrate both visualization

and image annotation into the existing framework (NIH proposal)

Hosts the image and sensor data of CenSSIS (>500 images online) http://censsis-db1.ece.neu.edu/

Provides metadata indexed image searching Uses XML tags to allow for easy information interchange Evolved into a project-based management system, allowing

users to organize their data hierarchically Key issue: how do we develop collaboration tools that

increase the value of data stored in the database? Presently exploring how best to integrate both visualization

and image annotation into the existing framework (NIH proposal)

CenSSIS Image and Data Information Management CenSSIS Image and Data Information Management

Addressing key research barriers in computational efficiency, embedded computing and image/sensor data management

Exploiting Grid resources to enable new discovery in SSI applications

Producing a image/data repository and software-engineered Subsurface Sensing and Imaging Toolsets

Developing enabling tools targeting system-level projects • Near real-time reconstruction and visualization• Visualization of complex motion• Predicting motion in image data using database indexing

techniques

Addressing key research barriers in computational efficiency, embedded computing and image/sensor data management

Exploiting Grid resources to enable new discovery in SSI applications

Producing a image/data repository and software-engineered Subsurface Sensing and Imaging Toolsets

Developing enabling tools targeting system-level projects • Near real-time reconstruction and visualization• Visualization of complex motion• Predicting motion in image data using database indexing

techniques

MVT