Ultrasound Visualization PipelineA Survey

Å. Birkeland, V. Šoltészová, D. Hönigmann, O. H. Gilja, S. Brekke, T. Ropinski and I. Viola

Definition of VisualizationVisualization:

the use of (usually) computer graphics to reveal insight into data to a userto form a mental vision, image, or picture of (something not visible or present to the sight, or of an abstraction); to make visible to the mind or imagination [Oxford Engl. Dict., 1989]

computer graphics, but not photo-realistic renderingThe purpose of computing is insight, not numbers[R. Hamming, 1962]The purpose of visualization is insight, not pictures[B. Schneidermann, 1999]

Åsmund Birkeland, University of Bergen 2

Ultrasound Imaging ModalityNon-invasiveRelatively inexpensiveHigh resolution

SpatiallyTemporally

NoiseRandom Speckle

Difficult to interpret

Åsmund Birkeland, University of Bergen 3

Usage of Medical UltrasoundDiagnosticsIntraoperative imagingIntervention

Åsmund Birkeland, University of Bergen 4

Ultrasound Image Modality2D ultrasound3D ultrasound

3D freehand ultrasoundDedicated 3D ultrasound probe

4D ultrasoundFunctional imaging

DopplerB-flowStrainContrast enhanced US

Åsmund Birkeland, University of Bergen 5

Ultrasound Visualization – The PipelinePre-processing: Processing ultrasound data prior to segmentation, registration or renderingSegmentation: Extracting features from ultrasound dataRegistration: Combining ultrasound data with other types of medical imaging modalitiesRendering: Presenting ultrasound dataAugmented Reality: Combining ultrasound rendering with the real world

Åsmund Birkeland, University of Bergen 6

Rendering Augmented RealityRegistrationRegistrationPre-processing

Ultrasound Visualization - The Pipeline

Åsmund Birkeland, University of Bergen 7

Pre-processingScan-conversionReconstruction of freehand ultrasoundData Enhancement

Åsmund Birkeland, University of Bergen 8

Gee et al. Processing and Visualizing Three-Dimensional Ultrasound Data

Segmentation

Åsmund Birkeland, University of Bergen 9

Automated assessment of ovarian follicles using a novel three-dimensional ultrasound software

MagiCut – Interactive clipping for 3D ultrasound

Vessel Extraction from UltrasoundUltrasound Painting of Vascular Tree

Quickly extract 3D models live during examinationMinimize interactionUsing basic 2D B-mode ultrasound + tracking

Åsmund Birkeland, University of Bergen 10

Vessel Extraction from Ultrasound

RegistrationCombining two or more image modalities in the same reference frameRigid RegistrationNon-Rigid Registration

Åsmund Birkeland, University of Bergen 12

Olesch et al. Matching CT and Ultrasound data of the Liver by Landmark constrained Image Registration

Wiein et al Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention

RenderingSurface Rendering3D freehand rendering4D ultrasound

Åsmund Birkeland, University of Bergen 13

Fattal and Lischinski Variational Classification for Visualization of 3D Ultrasound Data

Volume: Set of Slices

Åsmund Birkeland, University of Bergen 14

Rendering – Volume Visualization

3D→2D, e.g., a density volume from CT etc.direct volume rendering(semi-transparent volume)

15Åsmund Birkeland, University of Bergen

Rendering – Transfer Functions

Åsmund Birkeland, University of Bergen 16

Hönigmann et al. Adaptive Design of a Global Opacity Transfer Function for Direct Volume Rendering of Ultrasound Data

Mapping data to color and opacityNon-regular data intensities for the same tissue

Rendering – Multi-ModalDopplerMRI / CTIncrease in image clutter

Åsmund Birkeland, University of Bergen 17

Viola et al. Illustrated Ultrasound for Multimodal Data Interpretation of Liver Examinations

Petersch et al. Blood flow in its context: Combining 3D B-Mode and Color doppler Ultrasonic Data

Rendering - Shading

Åsmund Birkeland, University of Bergen 18

Chromatic Depth Coloring

Ropinski et al. Interactive Volumetric Lighting Simulating Scattering and Shadowing

Šoltészová et al. Multi-Directional Occlusion shading

Ultrasound and Augmented Reality

Åsmund Birkeland, University of Bergen 19Åsmund Birkeland, University of Bergen 19

Conclusion

Åsmund Birkeland, University of Bergen 20

AcknowledgementsIllustrasound Project, VERDIKT, The Norwegian Research CouncilHelwig Hauser – VisGroup, University of Bergen, NorwayWolfgang Wein - White Lion Technologies, Munich, Germany

Åsmund Birkeland, University of Bergen 21

Questions?Thank you!

Åsmund Birkeland, University of Bergen 22