Embed Size (px)
Citation preview
#CMIMI18#CMIMI18
Dr. Peter Chang, MDPresented By: Chanon Chantaduly
University of California Irvine
Hybrid 3D/2D Fully Convolutional Network Design for Volumetric Medical Datasets
#CMIMI18
Challenges with High-Res Input Data
#CMIMI18
Pros/Cons of Available Architectures
• 2D Only
• Easy and fast
• Lose 3D spatial information
• 3D only
• Slow and inefficient
• Too much spatial information
CNN
one slice
entire volume
CNN one slice
entire volume
#CMIMI18
Hybrid 3D/2D Architecture
• Intuition
• Takes in a 3D input slice and predicts a 2D slice
• Gives enough 3D information for a more accurate prediction
• Takes advantage of the fact that convolutional neural networks (CNN) can take in any amount of slices
slab
one sliceCNN
#CMIMI18
Hybrid 3D/2D Architecture
• Architecture
• Subsample X/Y with a ‘SAME’ padding convolutional operation of stride 2 (1x2x2)
• Subsample Z with a ‘VALID’ padding convolutional operation of stride 1 (2x1x1)
2x1x1
Z = 4Z = 5
X = 256Y = 256
1x2x2
X = 128
Y = 128‘SAME’
‘VALID’
stride 2
stride 1
#CMIMI18
Hybrid 3D/2D Architecture
• Architecture
• Residual projection operation to match the 3D arm with 2D arm
Z = 5
‘VALID’
5x1x1 Z = 1
+
From the left 3D Contracting Arm
From the right 2D Expanding Arm
Z = 1
stride 1
#CMIMI18
Hybrid 3D/2D Architecture
• Inference
• Pass in the entire volume, pad the top and bottom, then predict block by block
Z = 5predict prediction slab
slice we want to predict
padding
entire volume
#CMIMI18
Hybrid 3D/2D Architecture
• Inference
• Pass in the entire volume, pad the top and bottom, then predict block by block
Z = 5 predict
entire volume
padding
prediction slab
slice we want to predict
#CMIMI18
Hybrid 3D/2D Architecture
• Inference
• Pass in the entire volume, pad the top and bottom, then predict block by block
Z = 5
predict
entire volume
slice we want to predict
padding
prediction slab
#CMIMI18
U-Net Hybrid 3D/2D Architecture
#CMIMI18
FP Hybrid 3D/2D Architecture
#CMIMI18
Results and Conclusion
• Outperformed the corresponding 2D version in all datasets tested
• Huge gain in detecting small findings and the edges of top/bottom slices
• Slight increase in training time and no significant increase in inference time
#CMIMI18
Summary
• Any CNN with a contracting-expanding (encoder-decoder) structure can take in a 3D input of an arbitrary size to give a 2D output
• Subsample in the X/Y direction with a 1x2x2 with ‘SAME’ padding and a stride of 2
• Subsample in the Z direction with a 2x1x1 with ‘VALID’ padding and a stride of 1
• Residual projection operation
#CMIMI18
Questions?
Chanon [email protected]
(714) 509-2524
Center for AI in Diagnostic MedicineUC Irvine Medical Centerhttp://caidm.som.uci.edu/@TheCAIDM
