Database-Assisted Low-Dose CT Image

Restoration

Klaus MuellerComputer Science

Lab for Visual Analytics and Imaging (VAI)

Stony Brook University

Wei Xu, Sungsoo Ha and Klaus Mueller

Motivation

Low-dose CT:

* Images from Google.com

Motivation

Minimize the radiation, while maximize the clarity

Enforce better quality directly in the reconstruction process

• TV-CBCT [J. Xun & S. Jiang]

• ASD-POCS [E. Sidky & X. Pan]

• R-OS-SIRT [W. Xu & K. Mueller]

Solutions

Improve quality in a post-processing de-noising step

• [Z. Kelm et al.]

• [H. Yu & G. Wang]

• [J. Ma & Z. Liang]

• [W. Xu & K. Mueller]

Post-processing De-noising Filter - NLM

Neighborhood filters – Non-local Means (NLM)

• To update pixel x: a mean value of pixels in its search window

• Weight: by the patch similarity

y

x

Search Window W

Central pixel x x’s patch area Px

pixel y inside W y’s patch area Py

Assumption: there exists a high degree of redundancy to overcome noise by consulting similar patches to average contributions for a more stable outcome

Post-processing De-noising Filter - NLM

Neighborhood filters – Non-local Means (NLM)

• x,y,z: spatial variables W : search window, P : patch area around each pixel h : parameter to control the smoothness Ga: Gaussian kernel

x

x

Wy Pttytxa

Wyy

Pttytxa

xhpptG

phpptG

p)/)(exp(

)/)(exp(

22

22

'

y

x

NLM’s Results

Reduce moderate artifacts

Input NLM

NLM’s Results

But limited for extreme low-dose situation

Input NLM TVM

What to do now…

Information in the input image is not sufficient

Extend the search space beyond the current image• Utilize prior scans of the same patient:

- Z. Kelm, H. Yu & G. Wang, Q. Xu & G. Wang, J. Ma & Z. Liang, W. Xu & K. Mueller

- simple, but limited• Utilize database of different patients - find reference image and incorporate into the de-noising

Reference-based NLM (R-NLM)

Compare between central patch and the reference patch

Input Ref

weight, pixel value

y

x

22

22

exp( ( ) / )

exp( ( ) / )

y t

x

y t

x

crp crpa x t y

y W t Px

crpa x t

y W t P

G t p p h p

pG t p p h

R-NLM’s Result

Input NLM R-NLM

Gold Standard

Magic ?

But…

Matched Reference-based NLM (MR-NLM)

Input Matched-R Clean-R

pixel valueweight

22

22

exp( ( ) / )

exp( ( ) / )

y t

x

y t

x

drp crpa x t y

y W t Px

drpa x t

y W t P

G t p p h p

pG t p p h

MR-NLM’s Result

Input NLM MR-NLM

Magic ?

Yes !

Gold Standard

Refinement to MR-NLM

The refinement to NLM is also applicable to MR-NLM

Implement two redundancy control methods• Reduce search window redundancy [T. Tasdizen]: discard unrelated pixels whose mean and variance are different enough• Reduce patch redundancy [P. Coupe et al.]: apply PCA to high-D patch space project patches to a lower dimensional sub-space

Improve not only efficiency but also accuracy

Database-Assisted CT Image Restoration (DA-CTIR) Framework

Online Database Construction

2D Image Space High-D Image Feature Space

Image Scan Global Image Feature

Exact as salient local image structure and contextual information

Learn the cluster centers of the local features of all images and label them

Concatenate local labels to form global descriptor as distinct salient properties of the image

Local Image Feature Descriptor

In MR-NLM:• Input image is low-dose• The database contains only high (normal)-dose images • Matching is between artifact-free and artifact-contained

ones local feature descriptor should be tolerant to artifact (streak, noise, etc.) and small deformation

Scale-Invariant Feature Transform (SIFT) feature• Captures histogram of edge orientations in a local

neighborhood• Scale-invariant, transform-invariant and less sensitive to

noise

Local Image Feature Descriptor

SIFT feature descriptor:• Over the neighborhood of size 1616 dividing to 44 blocks• In each block, 8-orientation histogram of edges is

computed

Dense SIFTs over a regular spaced grid: better, robust• Grid spacing of 8 pixels, N = 3232 (6464) SIFTs for 2562

(5122) image

block

8-bin orientation histogram

neighborhood

448 128-D feature vector

Learn visual words

• To describe one image, the dimension is reduced from 128•N to N (N 1024 or 4096).

A set of local features {S0, S1, .., SN-1}

k-means

clustering

K cluster centers as visual words {V0, V1,

…, VK-1} as visual vocabulary V

Local feature vector is assigned to index of

the closest visual word

Labeling

Global Image Feature Descriptor

• Partition image to multi-resolution to increase the precision• Concatenate histograms of labels from each sub-region.• Totally, 26•K dimensions (K 50 in this paper)

A set of labels in fixed grid

positions

Spatial pyramid based vector quantization

Global Image

Feature

Dimension

2D Image Space High-D Image Feature Space

Scan Image Global Image Feature

128k-D per image

1k-D per image

1.3k-D per image

64k-D

Online Prior Search

2D Image Space High-D Image Feature Space

Target Image M nearest references

Support Kd-tree structure (PKD-tree) for fast labeling process, check our paper for details

Histogram Intersection

Essentially concatenated histograms while not only high-D vector; histogram intersection vs. Euclidean distance

Online De-noising

Registration FBP

De-noised image

MR-NLM

Target image, M nearest references, Low-dose condition

SIFT-flow• Tolerant to noise and

small deformation• Optical-flow to

obtain displacement field

• SIFT instead of pixel

Refined MR-NLM• Two redundancy

controls• Fall back to regular NLM

for pixel with close to zero normalization factor

Experiments

Two image databases (not pre-aligned):• 48 2562 head scans - 15 NIH visible human head images - 33 CT cadaver head images• 150 5122 human lung scans from two patients - “give a scan” online database

Original reconstructions are utilized as:• Basis for low-dose simulation (limited number of

projections with noise)• Basis for generating target scan (deformed or rotated and

then reconstruct with low-dose condition)• Gold standard for evaluation

Fan-beam geometry

Results

Head database: low-dose condition: 45-proj SNR 15

Ideal

Input

PriorsDA-CTIR

Refined DA-CTIR

Results

Lung database: low-dose condition: 60-proj SNR 20

Ideal

Input

DA-CTIR

Refined DA-CTIR

• 

• 

Future Works

PCA reduction to global image feature

Larger database for more experiments to verify effectiveness

GPU acceleration

Questions?

DA-CTIR