U-Net Fixed Point Quantization For MedicalImage Segmentation

MohammadHossein AskariHemmatSina Honari

Lucas RouhierChristian S. PeroneJulien Cohen-Adad

Yvon SavariaJean-Pierre David

MICCAI2019, October 17th

Presentation outline:

1 Medical Image Segmentation With Deep Neural Networks(DNNs)

2 Quantization in Deep Neural Networks

3 U-Net Fixed-Point Quantization for Medical ImageSegmentation

4 Results

5 Conclusion

U-Net Fixed Point Quantization For Medical Image Segmentation 2

3/24

Medical Image Segmentation With Deep Neural Networks (DNNs)

• Medical Image Segmentation are performed on MRI imagesto help doctors diagnose diseases.

Figure: Segmentation of a brain tumor in an MRI image [M.Havaie et al 2016].

U-Net Fixed Point Quantization For Medical Image Segmentation 3

4/24

Medical Image Segmentation With Deep Neural Networks (DNNs)

• Medical health centers and hospitals are equipped withpre-trained models used in medical CADs to analyse MRIimages.

• However, to perform these tasks with DNNs, a lot ofcomputation needs to be done.

• But how many operations are really needed?

U-Net Fixed Point Quantization For Medical Image Segmentation 4

5/24

Computation Cost in Deep Neural Networks (DNNs)

Training Computation Cost :Finishing a 90-epoch ImageNet-1k training with ResNet-50 on a NVIDIAM40 GPU takes 14 days. This training requires 1018 single precisionoperations in total [Y. You et al ”ImageNet Training in Minutes”].

Inference Computation Cost :Finishing a full pass of Imagenet with input size of 224x224 with batch sizeof 128 reuqires 13 GB feature memory and 497 GFLOPs [S. AlbanieGitHub:convnet-burden].

U-Net Fixed Point Quantization For Medical Image Segmentation 5

6/24

Cost and energy consumption source in DNNs:

• Number of parameters in the network (Data Movement).• Per operation computation cost.

Figure: Deep compression [H. Song, et al., 2015]

U-Net Fixed Point Quantization For Medical Image Segmentation 6

7/24

Quantization For Accelerating Computation in DNNs :

One way to accelerate computation and save energy in a DNN isto use less precision for computation:

• Less number of bits are required to be stored in memory,which reduces data movement.

• Floating point calculation is costly and slow. Less energyand computation will be spent per operation.

U-Net Fixed Point Quantization For Medical Image Segmentation 7

8/24

Quantization For Accelerating Computation in DNNs :

What is Quantization in DNN?Quantization is a technique to reduce memory consumption and thecomputation time of deep neural networks by lowering the precision ofparameters.

U-Net Fixed Point Quantization For Medical Image Segmentation 8

9/24

Quantization For Accelerating Computation in DNNs :

• Quantization not only uses less memory but it is moreenergy efficient:

Figure: Energy consumption of multiplication and accumulationin a 45nm process (Horowitz, 2014)

• In Intel Core i7 4770 3.40GHz, 32-bit multiplication is morethan 3 times faster for fixed point data compared tofloating point data. [https://goo.gl/7Y7GWt].

U-Net Fixed Point Quantization For Medical Image Segmentation 9

10/24

Quantization For Accelerating Computation in DNNs :

Does it Work?

U-Net Fixed Point Quantization For Medical Image Segmentation 10

11/24

Quantization For Accelerating Computation in DNNs :

• In [M. Courbariaux et al. ”BinaryConnect”] it has beenshown that for small models (MNIST size) using even 1-bitfor weights results in test accuracy drop of only 1%.

• For bigger models (like ImageNet) using 8-bit integersinstead of 32-bit floating point shows state-of-the-artperformance.

• Pytorch just added quantization option!

U-Net Fixed Point Quantization For Medical Image Segmentation 11

12/24

Existing works for quantization of medical images:

• FCN Quantization [Xu, X., et al. CVPR 2018]: Appliedquantization on Fully Convolutional Networks forbiomedical application.

• TernaryNet [Heinrich, M.P et al, IJCARS 2017]: Firstquantization results for U-Net using ternary net.

U-Net Fixed Point Quantization For Medical Image Segmentation 12

13/24

Our research objective:

Quantization for Medical Image Segmentation:In this work, we wanted to know does quantization works for life threateningtasks such as medical imaging?

• We wanted to use a well known model that is widely usedfor medical imaging.

• We wanted to have only fixed point operations so that itcan be used with an integer processors.

U-Net Fixed Point Quantization For Medical Image Segmentation 13

14/24

U-Net Fixed-Point Quantization for Medical ImageSegmentation

• In our work, we transformed all floating point computationto fixed-point computation.

xf = abs(x) − floor(abs(x)), xi = floor(abs(x)) (1)

quantize(x , n) = (round(clamp(x , n) << n)) >> n (2)

clamp(x , n) =

2n − 1 when x ≥ 2n − 1

x when 0 < x < 2n − 1

0 when x ≤ 0

(3)

• Qp i .f : fixed point quantization of parameter p by using ibits to represent the integer part and f bits to represent thefractional part.U-Net Fixed Point Quantization For Medical Image Segmentation 14

15/24

Challenges for Training:

• Dropout and Quantization: we found that when Dropout isapplied along with quantization, the accuracy drops a lot:

U-Net Fixed Point Quantization For Medical Image Segmentation 15

16/24

Challenges for Training:

• Keeping the last layer in full precision has much moreimpact than keeping the first layer in full precision.

• At inference, we used Pytorch batch-norm folding.Effectively including batch-norm parameters in thequantized model as part of the quantized weights.

U-Net Fixed Point Quantization For Medical Image Segmentation 16

17/24

Results For Quantization of U-Net Model for Medical Image:• We used three different datasets:

NIH

Pan

crea

sG

M S

pina

l Cor

dE

M D

roso

phila

Bra

in

Figure: Segments in show false positive, segments in show false negative and segments inshow true positive.

U-Net Fixed Point Quantization For Medical Image Segmentation 17

18/24

Results For Quantization of U-Net Model for Medical Image:

• Qa6.0,Qw0.4 compared to other methods:

Figure: shows best score overall and shows best score between three quantiation methods.

U-Net Fixed Point Quantization For Medical Image Segmentation 18

19/24

Results and Discussion:

• We found out that instead of using 32-bit floating pointvalues, we can use 4 bits for weights and 6 bits foractivations.

U-Net Fixed Point Quantization For Medical Image Segmentation 19

20/24

Results and Discussion:

• Fully fixed point data path:

CONV +ak-1

wt

bt

ak

Quant

Quant

Fixed Point Fixed Point

Fixed Point Fixed PointFixed Point

CONV +ak-1

wt

bt

ak

Ternary

Ternary

Ternary Float

Ternary Float

Ternary

Tern

aryN

etO

ur M

etho

d

U-Net Fixed Point Quantization For Medical Image Segmentation 20

21/24

Results and Discussion

• Relu Vs Tanh performance:

Figure: Comparing ReLU and Tanh run time using Intel’sOpenVino [Deanne. D rt. al Release notes for intel R© 2019]

U-Net Fixed Point Quantization For Medical Image Segmentation 21

22/24

Our Code is on GitHub:https://github.com/hossein1387/U-Net-Fixed-Point-Quantization-for-Medical-Image-Segmentation.

U-Net Fixed Point Quantization For Medical Image Segmentation 22

23/24

Our Contribution in this work:

• We report the first fixed point quantization results on theU-Net architecture for the medical image segmentation taskand show that the current quantization methods availablefor U-Net are not efficient for the hardware commonlyavailable in the industry.

• We quantify the impact of fixed point quantization on theperformance of the U-Net model using three differentmedical imaging datasets.

• We report results comparable to a full precisionsegmentation model by using only 6 bits for activation and4 bits for weights, effectively reducing the weights size by afactor of 8x and the activation size by a factor of 5x.

U-Net Fixed Point Quantization For Medical Image Segmentation 23

24/24

Thank you for your attention!

U-Net Fixed Point Quantization For Medical Image Segmentation 24