Hotspot Detection via Attention-based Deep Layout Metric Learningbyu/papers/C106-ICCAD2020-Metric... · 2020. 10. 29. · Hotspot Detection via Attention-based Deep Layout Metric

Hotspot Detection via Attention-basedDeep Layout Metric Learning

Hao Geng1, Haoyu Yang1, Lu Zhang1, Jin Miao2,Fan Yang3, Xuan Zeng3, Bei Yu1

1The Chinese University of Hong Kong2Synopsys

3Fudan University

1 / 1

Biography

Hao GENGPh.D. candidateThe Chinese University of Hong Kong (CUHK)[email protected]

Hao is a year-4 Ph.D. student at the Department of ComputerScience and Engineering, under the supervision of Prof. Bei YUsince Fall 2017. His research interests include optimization, ma-chine learning and deep learning techniques in VLSI CAD. Hehas received one best paper award nomination from ASPDAC2019. Recently he has worked on design space exploration inVLSI CAD.

2 / 1

mailto:[email protected]

Moore’s Law to Extreme Scaling

1940 1950 1960 1970 1980 1990 2000 2010 2020

10,000,000,000

110

1001,000

10,000100,000

1,000,00010,000,000

100,000,0001,000,000,000

Invention of the Transistor

10 1 0.1 0.01

Year

Num

ber o

f Tra

nsis

tors

per I

nteg

rate

d C

ircui

t

Moore’s LawProcess Technology µmAAAB7nicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLElsLDERMIEL2VsW2LC7d9mdMyEXfoSNhcbY+nvs/DcucIWCL5nk5b2ZzMyLEiks+v63V9ja3tndK+6XDg6Pjk/Kp2cdG6eG8TaLZWweI2q5FJq3UaDkj4nhVEWSd6Pp7cLvPnFjRawfcJbwUNGxFiPBKDqpW+2rlKjqoFzxa/4SZJMEOalAjtag/NUfxixVXCOT1Npe4CcYZtSgYJLPS/3U8oSyKR3znqOaKm7DbHnunFw5ZUhGsXGlkSzV3xMZVdbOVOQ6FcWJXfcW4n9eL8XRTZgJnaTINVstGqWSYEwWv5OhMJyhnDlCmRHuVsIm1FCGLqGSCyFYf3mTdOq1wK8F9/VKs5HHUYQLuIRrCKABTbiDFrSBwRSe4RXevMR78d69j1VrwctnzuEPvM8fNUyOxg==AAAB7nicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLElsLDERMIEL2VsW2LC7d9mdMyEXfoSNhcbY+nvs/DcucIWCL5nk5b2ZzMyLEiks+v63V9ja3tndK+6XDg6Pjk/Kp2cdG6eG8TaLZWweI2q5FJq3UaDkj4nhVEWSd6Pp7cLvPnFjRawfcJbwUNGxFiPBKDqpW+2rlKjqoFzxa/4SZJMEOalAjtag/NUfxixVXCOT1Npe4CcYZtSgYJLPS/3U8oSyKR3znqOaKm7DbHnunFw5ZUhGsXGlkSzV3xMZVdbOVOQ6FcWJXfcW4n9eL8XRTZgJnaTINVstGqWSYEwWv5OhMJyhnDlCmRHuVsIm1FCGLqGSCyFYf3mTdOq1wK8F9/VKs5HHUYQLuIRrCKABTbiDFrSBwRSe4RXevMR78d69j1VrwctnzuEPvM8fNUyOxg==AAAB7nicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLElsLDERMIEL2VsW2LC7d9mdMyEXfoSNhcbY+nvs/DcucIWCL5nk5b2ZzMyLEiks+v63V9ja3tndK+6XDg6Pjk/Kp2cdG6eG8TaLZWweI2q5FJq3UaDkj4nhVEWSd6Pp7cLvPnFjRawfcJbwUNGxFiPBKDqpW+2rlKjqoFzxa/4SZJMEOalAjtag/NUfxixVXCOT1Npe4CcYZtSgYJLPS/3U8oSyKR3znqOaKm7DbHnunFw5ZUhGsXGlkSzV3xMZVdbOVOQ6FcWJXfcW4n9eL8XRTZgJnaTINVstGqWSYEwWv5OhMJyhnDlCmRHuVsIm1FCGLqGSCyFYf3mTdOq1wK8F9/VKs5HHUYQLuIRrCKABTbiDFrSBwRSe4RXevMR78d69j1VrwctnzuEPvM8fNUyOxg==AAAB7nicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLElsLDERMIEL2VsW2LC7d9mdMyEXfoSNhcbY+nvs/DcucIWCL5nk5b2ZzMyLEiks+v63V9ja3tndK+6XDg6Pjk/Kp2cdG6eG8TaLZWweI2q5FJq3UaDkj4nhVEWSd6Pp7cLvPnFjRawfcJbwUNGxFiPBKDqpW+2rlKjqoFzxa/4SZJMEOalAjtag/NUfxixVXCOT1Npe4CcYZtSgYJLPS/3U8oSyKR3znqOaKm7DbHnunFw5ZUhGsXGlkSzV3xMZVdbOVOQ6FcWJXfcW4n9eL8XRTZgJnaTINVstGqWSYEwWv5OhMJyhnDlCmRHuVsIm1FCGLqGSCyFYf3mTdOq1wK8F9/VKs5HHUYQLuIRrCKABTbiDFrSBwRSe4RXevMR78d69j1VrwctnzuEPvM8fNUyOxg==µmAAAB7nicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLElsLDERMIEL2VsW2LC7d9mdMyEXfoSNhcbY+nvs/DcucIWCL5nk5b2ZzMyLEiks+v63V9ja3tndK+6XDg6Pjk/Kp2cdG6eG8TaLZWweI2q5FJq3UaDkj4nhVEWSd6Pp7cLvPnFjRawfcJbwUNGxFiPBKDqpW+2rlKjqoFzxa/4SZJMEOalAjtag/NUfxixVXCOT1Npe4CcYZtSgYJLPS/3U8oSyKR3znqOaKm7DbHnunFw5ZUhGsXGlkSzV3xMZVdbOVOQ6FcWJXfcW4n9eL8XRTZgJnaTINVstGqWSYEwWv5OhMJyhnDlCmRHuVsIm1FCGLqGSCyFYf3mTdOq1wK8F9/VKs5HHUYQLuIRrCKABTbiDFrSBwRSe4RXevMR78d69j1VrwctnzuEPvM8fNUyOxg==AAAB7nicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLElsLDERMIEL2VsW2LC7d9mdMyEXfoSNhcbY+nvs/DcucIWCL5nk5b2ZzMyLEiks+v63V9ja3tndK+6XDg6Pjk/Kp2cdG6eG8TaLZWweI2q5FJq3UaDkj4nhVEWSd6Pp7cLvPnFjRawfcJbwUNGxFiPBKDqpW+2rlKjqoFzxa/4SZJMEOalAjtag/NUfxixVXCOT1Npe4CcYZtSgYJLPS/3U8oSyKR3znqOaKm7DbHnunFw5ZUhGsXGlkSzV3xMZVdbOVOQ6FcWJXfcW4n9eL8XRTZgJnaTINVstGqWSYEwWv5OhMJyhnDlCmRHuVsIm1FCGLqGSCyFYf3mTdOq1wK8F9/VKs5HHUYQLuIRrCKABTbiDFrSBwRSe4RXevMR78d69j1VrwctnzuEPvM8fNUyOxg==AAAB7nicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLElsLDERMIEL2VsW2LC7d9mdMyEXfoSNhcbY+nvs/DcucIWCL5nk5b2ZzMyLEiks+v63V9ja3tndK+6XDg6Pjk/Kp2cdG6eG8TaLZWweI2q5FJq3UaDkj4nhVEWSd6Pp7cLvPnFjRawfcJbwUNGxFiPBKDqpW+2rlKjqoFzxa/4SZJMEOalAjtag/NUfxixVXCOT1Npe4CcYZtSgYJLPS/3U8oSyKR3znqOaKm7DbHnunFw5ZUhGsXGlkSzV3xMZVdbOVOQ6FcWJXfcW4n9eL8XRTZgJnaTINVstGqWSYEwWv5OhMJyhnDlCmRHuVsIm1FCGLqGSCyFYf3mTdOq1wK8F9/VKs5HHUYQLuIRrCKABTbiDFrSBwRSe4RXevMR78d69j1VrwctnzuEPvM8fNUyOxg==AAAB7nicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLElsLDERMIEL2VsW2LC7d9mdMyEXfoSNhcbY+nvs/DcucIWCL5nk5b2ZzMyLEiks+v63V9ja3tndK+6XDg6Pjk/Kp2cdG6eG8TaLZWweI2q5FJq3UaDkj4nhVEWSd6Pp7cLvPnFjRawfcJbwUNGxFiPBKDqpW+2rlKjqoFzxa/4SZJMEOalAjtag/NUfxixVXCOT1Npe4CcYZtSgYJLPS/3U8oSyKR3znqOaKm7DbHnunFw5ZUhGsXGlkSzV3xMZVdbOVOQ6FcWJXfcW4n9eL8XRTZgJnaTINVstGqWSYEwWv5OhMJyhnDlCmRHuVsIm1FCGLqGSCyFYf3mTdOq1wK8F9/VKs5HHUYQLuIRrCKABTbiDFrSBwRSe4RXevMR78d69j1VrwctnzuEPvM8fNUyOxg==

40048086

286386

486 Pentium

Pentium IIPentium 4

Core 2 DuoCore i7

Doubles every 2.1 yrs

A7

A10 A11

A12

Intel MicroprocessorsApple Microprocessors

A13

3 / 1

Manufacturability Status & Challenges

Dispearance

1980 1990 2000 2010 2020

10

1

0.1

um

[Courtesy Intel]

X

4 / 1

Challenge: Failure (Hotspot) Detection

Pre-OPC Layout Post-OPC Mask Hotspot on Wafer

I RET: OPC, SRAF, MPLI Still hotspot: low fidelity patternsI Simulations: extremely CPU intensive

Ra#o

%of%lith

ograph

y%sim

ula#

on%#me%

(normalize

d%by%40n

m%nod

e)%

Technology%node

Required(computa/onal(/me(reduc/on!

5 / 1

Hotspot Detection Hierarchy

Increasing verificationaccuracy

Sampling

Hotspot Detection

Lithography Simulation

(Relative) CPU runtime at each level

I Sampling (DRC Checking):scan and rule check each region

I Hotspot Detection:verify the sampled regions and report potential hotspots

I Lithography Simulation:final verification on the reported hotspots

6 / 1

Pattern Matching based Hotspot Detection

library'hotspot&

Pa)ern'matching'

hotspot&hotspot&

detected

hotspot&

undetected

detectedCannot&detect&hotspots&not&in&the&library&

I Fast and accurateI [Yao+,ICCAD’06], [Lin+,DAC’2013], [Wen+,TCAD’14], [Tseng+,SPIE’19]I Hard to detect non-seen pattern

7 / 1

Conventional Machine Learning based Hotspot Detection

Non$Hotspot

Hotspot

Hotspot&detec*on&model&

Classifica*on&

Extract&layout&features&

Hard,to,trade$off,accuracy,and,false,alarms,

I Train a model to predict new patternsI Decision-tree, ANN, SVM, Boosting ...I [Ding+,TCAD’11],[Yu+,JM3’15],[Zhang+,ICCAD’16], [Ye+,DATE’19]I Hard to balance accuracy and false-alarm

8 / 1

Deep Learning based Hotspot Detection

…

Hotspot

Non-Hotspot

Convolution + ReLU Layer Max Pooling Layer Full Connected Node

26664

C11,1 C12,1 C13,1 . . . C1n,1C21,1 C22,1 C23,1 . . . C2n,1

......

.... . .

...Cn1,1 Cn2,1 Cn3,1 . . . Cnn,1

37775

26664

C11,k C12,k C13,k . . . C1n,kC21,k C22,k C23,k . . . C2n,k

......

.... . .

...Cn1,k Cn2,k Cn3,k . . . Cnn,k

37775

(k

W λ α k y∗h y∗n

j ← 0 W > 0

j ← j + 1m F1 F2 Fm

i← 1, 2, ..., mGi ← (Fi)

Ḡ ← 1m∑m

i=1 GiW←W − λḠ

j mod k = 0λ← αλ j ← 0

f

W λα ∈ (0, 1) k

y∗h y∗n

jm F1

F2 FmGi

WλḠ γ

j k λαλ k

0 100 200 300 400 500

0.00

1.00

2.00

3.00

nc

y∗ ∈ Rn y∗y∗(c) = 1

∑ni=1 y

∗(i) = 1 y∗

y

y∗ = y∗n = [1, 0]y∗ = y∗h = [0, 1]

x = [xh, xn](0, 1)

y(0) =exp xh

exp xh + exp xn, y(1) =

exp xnexp xh + exp xn

,

TCAD’19

DAC’19

0278-0070 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCAD.2019.2912948, IEEETransactions on Computer-Aided Design of Integrated Circuits and Systems

Input/12x12x32

Conv/3x3x16

Conv/3x3x16

MaxPool/2x2

Conv/3x3x32Conv/3x3x32

Conv/3x3x32Conv/3x3x32

MaxPool/2x2MaxPool/2x2

FC/250FC/250

FC/2FC/2

SoftmaxSoftmax

Pairwise contrastive

loss

Weighted cross entropy

ẑiAAADUnicfZLPbtQwEMa9CX9KKNDCkYtFisRplWQP7bFSLxyLxLaV4mjleCcbax0nsp3CNs1jcIWX4sKrcMJJU6mblo5k6dPMfP6NrUkrwbUJgj8Tx33y9NnznRfey91Xr9/s7b8902WtGMxZKUp1kVINgkuYG24EXFQKaJEKOE/XJ139/BKU5qX8ajYVJAVdSZ5xRo1NxQckp6a5ahf8YLHnB9OgD3xfhIPw0RCni30nJMuS1QVIwwTVOg6DyiQNVYYzAa1Hag0VZWu6gthKSQvQSdPP3OKPNrPEWanskQb32buOhhZab4rUdhbU5Hpc65IP1eLaZEdJw2VVG5DsBpTVApsSdx+Al1wBM2JjBWWK21kxy6mizNhv8jyPSPjGyqKgctmQVNE2DpOGgNS1go7WEAGZIYLKlQA/JIqvcnPdttvGNZiHjde3FqL6G8ZGS2S9OXqMislllVNpyqLxo3aYAfc9U+xHd8th+yjPXq74dxBtPPs/EI+I/qzFt9CO1j8M+7Mtrh0Mj9F2ycLxSt0XZ9E0DKbhl8g/PhrWbQe9Rx/QJxSiQ3SMPqNTNEcMlegH+ol+Ob+dv+7EdW9ancngeYe2wt39B+h5FM8=AAADUnicfZLPbtQwEMa9CX9KKNDCkYtFisRplWQP7bFSLxyLxLaV4mjleCcbax0nsp3CNs1jcIWX4sKrcMJJU6mblo5k6dPMfP6NrUkrwbUJgj8Tx33y9NnznRfey91Xr9/s7b8902WtGMxZKUp1kVINgkuYG24EXFQKaJEKOE/XJ139/BKU5qX8ajYVJAVdSZ5xRo1NxQckp6a5ahf8YLHnB9OgD3xfhIPw0RCni30nJMuS1QVIwwTVOg6DyiQNVYYzAa1Hag0VZWu6gthKSQvQSdPP3OKPNrPEWanskQb32buOhhZab4rUdhbU5Hpc65IP1eLaZEdJw2VVG5DsBpTVApsSdx+Al1wBM2JjBWWK21kxy6mizNhv8jyPSPjGyqKgctmQVNE2DpOGgNS1go7WEAGZIYLKlQA/JIqvcnPdttvGNZiHjde3FqL6G8ZGS2S9OXqMislllVNpyqLxo3aYAfc9U+xHd8th+yjPXq74dxBtPPs/EI+I/qzFt9CO1j8M+7Mtrh0Mj9F2ycLxSt0XZ9E0DKbhl8g/PhrWbQe9Rx/QJxSiQ3SMPqNTNEcMlegH+ol+Ob+dv+7EdW9ancngeYe2wt39B+h5FM8=AAADUnicfZLPbtQwEMa9CX9KKNDCkYtFisRplWQP7bFSLxyLxLaV4mjleCcbax0nsp3CNs1jcIWX4sKrcMJJU6mblo5k6dPMfP6NrUkrwbUJgj8Tx33y9NnznRfey91Xr9/s7b8902WtGMxZKUp1kVINgkuYG24EXFQKaJEKOE/XJ139/BKU5qX8ajYVJAVdSZ5xRo1NxQckp6a5ahf8YLHnB9OgD3xfhIPw0RCni30nJMuS1QVIwwTVOg6DyiQNVYYzAa1Hag0VZWu6gthKSQvQSdPP3OKPNrPEWanskQb32buOhhZab4rUdhbU5Hpc65IP1eLaZEdJw2VVG5DsBpTVApsSdx+Al1wBM2JjBWWK21kxy6mizNhv8jyPSPjGyqKgctmQVNE2DpOGgNS1go7WEAGZIYLKlQA/JIqvcnPdttvGNZiHjde3FqL6G8ZGS2S9OXqMislllVNpyqLxo3aYAfc9U+xHd8th+yjPXq74dxBtPPs/EI+I/qzFt9CO1j8M+7Mtrh0Mj9F2ycLxSt0XZ9E0DKbhl8g/PhrWbQe9Rx/QJxSiQ3SMPqNTNEcMlegH+ol+Ob+dv+7EdW9ancngeYe2wt39B+h5FM8=AAADUnicfZLPbtQwEMa9CX9KKNDCkYtFisRplWQP7bFSLxyLxLaV4mjleCcbax0nsp3CNs1jcIWX4sKrcMJJU6mblo5k6dPMfP6NrUkrwbUJgj8Tx33y9NnznRfey91Xr9/s7b8902WtGMxZKUp1kVINgkuYG24EXFQKaJEKOE/XJ139/BKU5qX8ajYVJAVdSZ5xRo1NxQckp6a5ahf8YLHnB9OgD3xfhIPw0RCni30nJMuS1QVIwwTVOg6DyiQNVYYzAa1Hag0VZWu6gthKSQvQSdPP3OKPNrPEWanskQb32buOhhZab4rUdhbU5Hpc65IP1eLaZEdJw2VVG5DsBpTVApsSdx+Al1wBM2JjBWWK21kxy6mizNhv8jyPSPjGyqKgctmQVNE2DpOGgNS1go7WEAGZIYLKlQA/JIqvcnPdttvGNZiHjde3FqL6G8ZGS2S9OXqMislllVNpyqLxo3aYAfc9U+xHd8th+yjPXq74dxBtPPs/EI+I/qzFt9CO1j8M+7Mtrh0Mj9F2ycLxSt0XZ9E0DKbhl8g/PhrWbQe9Rx/QJxSiQ3SMPqNTNEcMlegH+ol+Ob+dv+7EdW9ancngeYe2wt39B+h5FM8=

ŷiAAADUnicfZLPbtQwEMa9G/6UUKCFIxeLFInTKske6LESF45FYttKcbRyvJONtbYT2U5hleYxuMJLceFVOOGkqdRNS0ey9GlmPv/G1mSV4MaG4Z/J1Hv0+MnTvWf+8/0XL18dHL4+M2WtGSxYKUp9kVEDgitYWG4FXFQaqMwEnGebT139/BK04aX6arcVpJKuFc85o9alkiNSUNts2yU/Wh4E4SzsA98V0SACNMTp8nAakVXJagnKMkGNSaKwsmlDteVMQOuT2kBF2YauIXFSUQkmbfqZW/zeZVY4L7U7yuI+e9vRUGnMVmauU1JbmHGtS95XS2qbH6cNV1VtQbFrUF4LbEvcfQBecQ3Miq0TlGnuZsWsoJoy677J932i4BsrpaRq1ZBM0zaJ0oaAMrWGjtYQAbklgqq1gCAimq8Le9W2u8YN2PuNVzcWovsbxkZHZL05foiKyWVVUGVL2QRxO8yA+54ZDuLb5ah9kOcu1/w7iDaZ/x+IR8Rg3uIbaEfrH4aD+Q7XDYbHaLdk0Xil7oqzeBaFs+hLHJwcD+u2h96id+gDitBHdII+o1O0QAyV6Af6iX5Nf0//ehPPu26dTgbPG7QT3v4/5ZEUzg==AAADUnicfZLPbtQwEMa9G/6UUKCFIxeLFInTKske6LESF45FYttKcbRyvJONtbYT2U5hleYxuMJLceFVOOGkqdRNS0ey9GlmPv/G1mSV4MaG4Z/J1Hv0+MnTvWf+8/0XL18dHL4+M2WtGSxYKUp9kVEDgitYWG4FXFQaqMwEnGebT139/BK04aX6arcVpJKuFc85o9alkiNSUNts2yU/Wh4E4SzsA98V0SACNMTp8nAakVXJagnKMkGNSaKwsmlDteVMQOuT2kBF2YauIXFSUQkmbfqZW/zeZVY4L7U7yuI+e9vRUGnMVmauU1JbmHGtS95XS2qbH6cNV1VtQbFrUF4LbEvcfQBecQ3Miq0TlGnuZsWsoJoy677J932i4BsrpaRq1ZBM0zaJ0oaAMrWGjtYQAbklgqq1gCAimq8Le9W2u8YN2PuNVzcWovsbxkZHZL05foiKyWVVUGVL2QRxO8yA+54ZDuLb5ah9kOcu1/w7iDaZ/x+IR8Rg3uIbaEfrH4aD+Q7XDYbHaLdk0Xil7oqzeBaFs+hLHJwcD+u2h96id+gDitBHdII+o1O0QAyV6Af6iX5Nf0//ehPPu26dTgbPG7QT3v4/5ZEUzg==AAADUnicfZLPbtQwEMa9G/6UUKCFIxeLFInTKske6LESF45FYttKcbRyvJONtbYT2U5hleYxuMJLceFVOOGkqdRNS0ey9GlmPv/G1mSV4MaG4Z/J1Hv0+MnTvWf+8/0XL18dHL4+M2WtGSxYKUp9kVEDgitYWG4FXFQaqMwEnGebT139/BK04aX6arcVpJKuFc85o9alkiNSUNts2yU/Wh4E4SzsA98V0SACNMTp8nAakVXJagnKMkGNSaKwsmlDteVMQOuT2kBF2YauIXFSUQkmbfqZW/zeZVY4L7U7yuI+e9vRUGnMVmauU1JbmHGtS95XS2qbH6cNV1VtQbFrUF4LbEvcfQBecQ3Miq0TlGnuZsWsoJoy677J932i4BsrpaRq1ZBM0zaJ0oaAMrWGjtYQAbklgqq1gCAimq8Le9W2u8YN2PuNVzcWovsbxkZHZL05foiKyWVVUGVL2QRxO8yA+54ZDuLb5ah9kOcu1/w7iDaZ/x+IR8Rg3uIbaEfrH4aD+Q7XDYbHaLdk0Xil7oqzeBaFs+hLHJwcD+u2h96id+gDitBHdII+o1O0QAyV6Af6iX5Nf0//ehPPu26dTgbPG7QT3v4/5ZEUzg==AAADUnicfZLPbtQwEMa9G/6UUKCFIxeLFInTKske6LESF45FYttKcbRyvJONtbYT2U5hleYxuMJLceFVOOGkqdRNS0ey9GlmPv/G1mSV4MaG4Z/J1Hv0+MnTvWf+8/0XL18dHL4+M2WtGSxYKUp9kVEDgitYWG4FXFQaqMwEnGebT139/BK04aX6arcVpJKuFc85o9alkiNSUNts2yU/Wh4E4SzsA98V0SACNMTp8nAakVXJagnKMkGNSaKwsmlDteVMQOuT2kBF2YauIXFSUQkmbfqZW/zeZVY4L7U7yuI+e9vRUGnMVmauU1JbmHGtS95XS2qbH6cNV1VtQbFrUF4LbEvcfQBecQ3Miq0TlGnuZsWsoJoy677J932i4BsrpaRq1ZBM0zaJ0oaAMrWGjtYQAbklgqq1gCAimq8Le9W2u8YN2PuNVzcWovsbxkZHZL05foiKyWVVUGVL2QRxO8yA+54ZDuLb5ah9kOcu1/w7iDaZ/x+IR8Rg3uIbaEfrH4aD+Q7XDYbHaLdk0Xil7oqzeBaFs+hLHJwcD+u2h96id+gDitBHdII+o1O0QAyV6Af6iX5Nf0//ehPPu26dTgbPG7QT3v4/5ZEUzg==

ClusteringStream

ClassificationStream

Dropout 50% Dropout 50%

Fig. 3: The architecture of MTNN.

Labeled hotspots

Cluster0

xi

xj

di

dj

Cluster1

Learned boundary

Real boundary

Labeled non-hotspotsUnlabeled samples

Fig. 4: Illustration of how MTNN works.

clustering are jointly learned. The two streams share layers forfeature extraction at the early stages and then split into twobranches [29]. The shared layers include two convolutionallayers, one ReLU layer, and one max pooling layer. Thekernel sizes and the number of kernels are annotated in thefigure. The max pooling layer performs 2⇥ 2 downsampling.

For hotspot detection, the classification stream is used todo pseudo-labeling and the final hotspot detection, while theclustering stream is used to estimate labeling confidence withweights. Fig. 4 illustrates how the MTTNN works. The blueand orange dots are labeled hotspots and labeled nonhotspots

separately, the solid red line is the real boundary betweenthem. With jointly learned clustering and classification, wegot the green dash line as the learned boundary. The greysquare is the unlabeled samples. For unlabeled samples xiand xj , if we used the trained model to assign pseudo labels,sample xi is pseudo-labeled wrongly while sample xj ispseudo-labeled correctly. If we start the next training using thepseudo-labeled samples without computing weight throughclustering, then wrongly-labeled xi could compromise thefinal prediction performance. Clustering analyzes samples’distributions and divides samples into two clusters. Sampleswith the same labels tend to be in the same cluster due totheir similar distributions. A sample’s average distance tothe predicted cluster represents its labeling confidence. Thelonger the distance is, the less the labeling confidence is. Asshown in Fig 4, di > dj , it means that the wrongly pseudo-labeled sample xi has lower confidence than correctly pseudo-labeled sample xj . xi should have less influence on modeltraining. Therefore, lower weight is assigned to xi to alleviateits influence.

The individual layers for two streams are identical exceptthat the weights are learned separately and the loss functionsare different. The classification stream behaves as an ordinaryhotspot detector as in other neural network architectures. Itcan predict labels for unlabeled samples and its loss functionfor training is the weighted cross entropy where the weightscome along with data samples. The loss function is definedas follows,

Lclass = �NX

i=1

wi(

KX

k=1

yki log ŷki ), (3)

where N is the number of input samples, K is the number ofclasses, which is 2 in hotspot detection. Vectors yi and ŷi arethe one-hot encoding of actual class labels and the softmaxoutput for ith sample, respectively. Weight wi indicates theconfidence of the ith sample’s label. The weights for labeledsamples are set to 1, while the weights for unlabeled samplesare calculated during the training with the clustering stream.

The main target of clustering stream is to determine theweights of unlabeled samples based on their distances tothe predicted clusters. To further alleviate the influence ofuntrusted labeling, pairwise constraints [37] are introducedfor the loss function of clustering. A pairwise constraint isa pair of samples with the information of whether they aresimilar.

Definition 3 (Similar pair). If the labels of the two samplesare the same, they are a similar pair, vice versa.

For a labeled sample, its actual label is used. For anunlabeled sample, the pseudo-label predicted from the classi-fication stream is used.

Pairwise constraints can be generated by enumerating al-l pairs from the labeled or pseudo-labeled data samples.Clustering training with pairwise constraints enables bettertolerance to labeling error for unlabeled data. The outputof the final softmax layer in the clustering stream could beseen as the distribution probability over different clusters.

ASPDAC’19

I Success has been achieved: [Yang+,TCAD’19], [Chen+,ASPDAC’19], [Jiang+,DAC’19]I Two-stage flow and a lack of guidances from supervised informationI Question: Can we learn more informative and discriminative features?

9 / 1

Metric Learning in Daily Life

I Several applications of metric learning for similarity measures

Recognizing handwriting

Automatic detection of facesSearch Engines to match a query (could be text, image and etc.)

10 / 1

Basic Concepts of Metric Learning

DefinitionLearning a metric that quantify the similarity or the “distance” between every pair of samplesin a dataset.

I Pre-defined Metrics: Metrics which are fully specified without the knowledge of data.For example, Euclidian Distance.

D(x1, x2) = (x1 − x2)>(x1 − x2)

I Metrics which can only be defined with the knowledge of the data. For example,Mahalanobis Distance

D(x1, x2) = (x1 − x2)>M(x1 − x2)

11 / 1

Deep Metric Learning

Deep Metric Learning

Deep metric learning maps an image into a feature vector in a manifold space via deepneural networks. In this manifold space, the Euclidean distance (or the cosine distance)can be directly used as the distance metric between two points.

I Measuring the similarity among different layout designs is extremely crucialI Supervised Feature Extractor: learn feature embedding with label information to well

represent layoutsI Less manually design

12 / 1

Attention Mechanism: An Example in Computer Vision

Channel-wise Attention in Semantic Segmentation

7 / 13

Channel-wise Attention in Semantic Segmentation

7 / 13

=AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0EYI2FhYJGA+IDnC3mYuWbO3d+zuCeHIL7CxUMTWn2Tnv3GTXKGJDwYe780wMy9IBNfGdb+dwtr6xuZWcbu0s7u3f1A+PGrpOFUMmywWseoEVKPgEpuGG4GdRCGNAoHtYHw789tPqDSP5YOZJOhHdCh5yBk1Vmrc9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+aHTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZl+TAVfIjJhYQpni9lbCRlRRZmw2JRuCt/zyKmldVD236jUuK7X7PI4inMApnIMHV1CDO6hDExggPMMrvDmPzovz7nwsWgtOPnMMf+B8/gCPz4zLAAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0EYI2FhYJGA+IDnC3mYuWbO3d+zuCeHIL7CxUMTWn2Tnv3GTXKGJDwYe780wMy9IBNfGdb+dwtr6xuZWcbu0s7u3f1A+PGrpOFUMmywWseoEVKPgEpuGG4GdRCGNAoHtYHw789tPqDSP5YOZJOhHdCh5yBk1Vmrc9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+aHTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZl+TAVfIjJhYQpni9lbCRlRRZmw2JRuCt/zyKmldVD236jUuK7X7PI4inMApnIMHV1CDO6hDExggPMMrvDmPzovz7nwsWgtOPnMMf+B8/gCPz4zLAAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0EYI2FhYJGA+IDnC3mYuWbO3d+zuCeHIL7CxUMTWn2Tnv3GTXKGJDwYe780wMy9IBNfGdb+dwtr6xuZWcbu0s7u3f1A+PGrpOFUMmywWseoEVKPgEpuGG4GdRCGNAoHtYHw789tPqDSP5YOZJOhHdCh5yBk1Vmrc9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+aHTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZl+TAVfIjJhYQpni9lbCRlRRZmw2JRuCt/zyKmldVD236jUuK7X7PI4inMApnIMHV1CDO6hDExggPMMrvDmPzovz7nwsWgtOPnMMf+B8/gCPz4zLAAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0EYI2FhYJGA+IDnC3mYuWbO3d+zuCeHIL7CxUMTWn2Tnv3GTXKGJDwYe780wMy9IBNfGdb+dwtr6xuZWcbu0s7u3f1A+PGrpOFUMmywWseoEVKPgEpuGG4GdRCGNAoHtYHw789tPqDSP5YOZJOhHdCh5yBk1Vmrc9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+aHTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZl+TAVfIjJhYQpni9lbCRlRRZmw2JRuCt/zyKmldVD236jUuK7X7PI4inMApnIMHV1CDO6hDExggPMMrvDmPzovz7nwsWgtOPnMMf+B8/gCPz4zL

bicycle (enhance)

environment (weaken)

human (enhance)

13 / 1

Our Solution: Overview

Branch I: DeepMetric Learner

Input Layouts

Backbone Network

Branch II: HotspotClassificationFully Connected Attention Module Concat

LpredictionAAACAXicbVBNS8NAFHypX7V+Rb0IXhaL4KkkIuix6MWDhwq2FtoQNptNu3SzCbsboYR68a948aCIV/+FN/+NmzYHbR1YGGbesO9NkHKmtON8W5Wl5ZXVtep6bWNza3vH3t3rqCSThLZJwhPZDbCinAna1kxz2k0lxXHA6X0wuir8+wcqFUvEnR6n1IvxQLCIEayN5NsH/RjrIcE8v5n4uYmGjBTOxLfrTsOZAi0StyR1KNHy7a9+mJAspkITjpXquU6qvRxLzQink1o/UzTFZIQHtGeowDFVXj69YIKOjRKiKJHmCY2m6u9EjmOlxnFgJot91bxXiP95vUxHF17ORJppKsjsoyjjSCeoqAOFTFKi+dgQTCQzuyIyxBITbUqrmRLc+ZMXSee04ToN9/as3rws66jCIRzBCbhwDk24hha0gcAjPMMrvFlP1ov1bn3MRitWmdmHP7A+fwCYu5edAAACAXicbVBNS8NAFHypX7V+Rb0IXhaL4KkkIuix6MWDhwq2FtoQNptNu3SzCbsboYR68a948aCIV/+FN/+NmzYHbR1YGGbesO9NkHKmtON8W5Wl5ZXVtep6bWNza3vH3t3rqCSThLZJwhPZDbCinAna1kxz2k0lxXHA6X0wuir8+wcqFUvEnR6n1IvxQLCIEayN5NsH/RjrIcE8v5n4uYmGjBTOxLfrTsOZAi0StyR1KNHy7a9+mJAspkITjpXquU6qvRxLzQink1o/UzTFZIQHtGeowDFVXj69YIKOjRKiKJHmCY2m6u9EjmOlxnFgJot91bxXiP95vUxHF17ORJppKsjsoyjjSCeoqAOFTFKi+dgQTCQzuyIyxBITbUqrmRLc+ZMXSee04ToN9/as3rws66jCIRzBCbhwDk24hha0gcAjPMMrvFlP1ov1bn3MRitWmdmHP7A+fwCYu5edAAACAXicbVBNS8NAFHypX7V+Rb0IXhaL4KkkIuix6MWDhwq2FtoQNptNu3SzCbsboYR68a948aCIV/+FN/+NmzYHbR1YGGbesO9NkHKmtON8W5Wl5ZXVtep6bWNza3vH3t3rqCSThLZJwhPZDbCinAna1kxz2k0lxXHA6X0wuir8+wcqFUvEnR6n1IvxQLCIEayN5NsH/RjrIcE8v5n4uYmGjBTOxLfrTsOZAi0StyR1KNHy7a9+mJAspkITjpXquU6qvRxLzQink1o/UzTFZIQHtGeowDFVXj69YIKOjRKiKJHmCY2m6u9EjmOlxnFgJot91bxXiP95vUxHF17ORJppKsjsoyjjSCeoqAOFTFKi+dgQTCQzuyIyxBITbUqrmRLc+ZMXSee04ToN9/as3rws66jCIRzBCbhwDk24hha0gcAjPMMrvFlP1ov1bn3MRitWmdmHP7A+fwCYu5edAAACAXicbVBNS8NAFHypX7V+Rb0IXhaL4KkkIuix6MWDhwq2FtoQNptNu3SzCbsboYR68a948aCIV/+FN/+NmzYHbR1YGGbesO9NkHKmtON8W5Wl5ZXVtep6bWNza3vH3t3rqCSThLZJwhPZDbCinAna1kxz2k0lxXHA6X0wuir8+wcqFUvEnR6n1IvxQLCIEayN5NsH/RjrIcE8v5n4uYmGjBTOxLfrTsOZAi0StyR1KNHy7a9+mJAspkITjpXquU6qvRxLzQink1o/UzTFZIQHtGeowDFVXj69YIKOjRKiKJHmCY2m6u9EjmOlxnFgJot91bxXiP95vUxHF17ORJppKsjsoyjjSCeoqAOFTFKi+dgQTCQzuyIyxBITbUqrmRLc+ZMXSee04ToN9/as3rws66jCIRzBCbhwDk24hha0gcAjPMMrvFlP1ov1bn3MRitWmdmHP7A+fwCYu5ed

LmetricAAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIoMuiGxcuKthaaEOYTG/aoTNJmJkINQR/xY0LRdz6H+78GydtFtp6YOBwzr3cMydIOFPacb6tytLyyupadb22sbm1vWPv7nVUnEoKbRrzWHYDooCzCNqaaQ7dRAIRAYf7YHxV+PcPIBWLozs9ScATZBixkFGijeTbB31B9IgSnt3kfiZAS0Zz3647DWcKvEjcktRRiZZvf/UHMU0FRJpyolTPdRLtZURqRjnktX6qICF0TIbQMzQiApSXTdPn+NgoAxzG0rxI46n6eyMjQqmJCMxkkVXNe4X4n9dLdXjhZSxKUg0RnR0KU451jIsq8IBJoJpPDCFUMpMV0xGRhGpTWM2U4M5/eZF0Thuu03Bvz+rNy7KOKjpER+gEuegcNdE1aqE2ougRPaNX9GY9WS/Wu/UxG61Y5c4++gPr8wdWOpXIAAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIoMuiGxcuKthaaEOYTG/aoTNJmJkINQR/xY0LRdz6H+78GydtFtp6YOBwzr3cMydIOFPacb6tytLyyupadb22sbm1vWPv7nVUnEoKbRrzWHYDooCzCNqaaQ7dRAIRAYf7YHxV+PcPIBWLozs9ScATZBixkFGijeTbB31B9IgSnt3kfiZAS0Zz3647DWcKvEjcktRRiZZvf/UHMU0FRJpyolTPdRLtZURqRjnktX6qICF0TIbQMzQiApSXTdPn+NgoAxzG0rxI46n6eyMjQqmJCMxkkVXNe4X4n9dLdXjhZSxKUg0RnR0KU451jIsq8IBJoJpPDCFUMpMV0xGRhGpTWM2U4M5/eZF0Thuu03Bvz+rNy7KOKjpER+gEuegcNdE1aqE2ougRPaNX9GY9WS/Wu/UxG61Y5c4++gPr8wdWOpXIAAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIoMuiGxcuKthaaEOYTG/aoTNJmJkINQR/xY0LRdz6H+78GydtFtp6YOBwzr3cMydIOFPacb6tytLyyupadb22sbm1vWPv7nVUnEoKbRrzWHYDooCzCNqaaQ7dRAIRAYf7YHxV+PcPIBWLozs9ScATZBixkFGijeTbB31B9IgSnt3kfiZAS0Zz3647DWcKvEjcktRRiZZvf/UHMU0FRJpyolTPdRLtZURqRjnktX6qICF0TIbQMzQiApSXTdPn+NgoAxzG0rxI46n6eyMjQqmJCMxkkVXNe4X4n9dLdXjhZSxKUg0RnR0KU451jIsq8IBJoJpPDCFUMpMV0xGRhGpTWM2U4M5/eZF0Thuu03Bvz+rNy7KOKjpER+gEuegcNdE1aqE2ougRPaNX9GY9WS/Wu/UxG61Y5c4++gPr8wdWOpXIAAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIoMuiGxcuKthaaEOYTG/aoTNJmJkINQR/xY0LRdz6H+78GydtFtp6YOBwzr3cMydIOFPacb6tytLyyupadb22sbm1vWPv7nVUnEoKbRrzWHYDooCzCNqaaQ7dRAIRAYf7YHxV+PcPIBWLozs9ScATZBixkFGijeTbB31B9IgSnt3kfiZAS0Zz3647DWcKvEjcktRRiZZvf/UHMU0FRJpyolTPdRLtZURqRjnktX6qICF0TIbQMzQiApSXTdPn+NgoAxzG0rxI46n6eyMjQqmJCMxkkVXNe4X4n9dLdXjhZSxKUg0RnR0KU451jIsq8IBJoJpPDCFUMpMV0xGRhGpTWM2U4M5/eZF0Thuu03Bvz+rNy7KOKjpER+gEuegcNdE1aqE2ougRPaNX9GY9WS/Wu/UxG61Y5c4++gPr8wdWOpXI

Pooling

1 ⇥ 1AAAB8XicbVDLSgNBEOyNrxhfUY9eBoPgKeyIoMegF48RzAOTJcxOZpMhs7PLTK8QQv7CiwdFvPo33vwbJ8keNLGgoajqprsrTJW06PvfXmFtfWNzq7hd2tnd2z8oHx41bZIZLho8UYlph8wKJbVooEQl2qkRLA6VaIWj25nfehLGykQ/4DgVQcwGWkaSM3TSIyVdlLGwhPbKFb/qz0FWCc1JBXLUe+Wvbj/hWSw0csWs7VA/xWDCDEquxLTUzaxIGR+xgeg4qplbE0zmF0/JmVP6JEqMK41krv6emLDY2nEcus6Y4dAuezPxP6+TYXQdTKROMxSaLxZFmSKYkNn7pC+N4KjGjjBupLuV8CEzjKMLqeRCoMsvr5LmRZX6VXp/Wand5HEU4QRO4RwoXEEN7qAODeCg4Rle4c2z3ov37n0sWgtePnMMf+B9/gBHao/+AAAB8XicbVDLSgNBEOyNrxhfUY9eBoPgKeyIoMegF48RzAOTJcxOZpMhs7PLTK8QQv7CiwdFvPo33vwbJ8keNLGgoajqprsrTJW06PvfXmFtfWNzq7hd2tnd2z8oHx41bZIZLho8UYlph8wKJbVooEQl2qkRLA6VaIWj25nfehLGykQ/4DgVQcwGWkaSM3TSIyVdlLGwhPbKFb/qz0FWCc1JBXLUe+Wvbj/hWSw0csWs7VA/xWDCDEquxLTUzaxIGR+xgeg4qplbE0zmF0/JmVP6JEqMK41krv6emLDY2nEcus6Y4dAuezPxP6+TYXQdTKROMxSaLxZFmSKYkNn7pC+N4KjGjjBupLuV8CEzjKMLqeRCoMsvr5LmRZX6VXp/Wand5HEU4QRO4RwoXEEN7qAODeCg4Rle4c2z3ov37n0sWgtePnMMf+B9/gBHao/+AAAB8XicbVDLSgNBEOyNrxhfUY9eBoPgKeyIoMegF48RzAOTJcxOZpMhs7PLTK8QQv7CiwdFvPo33vwbJ8keNLGgoajqprsrTJW06PvfXmFtfWNzq7hd2tnd2z8oHx41bZIZLho8UYlph8wKJbVooEQl2qkRLA6VaIWj25nfehLGykQ/4DgVQcwGWkaSM3TSIyVdlLGwhPbKFb/qz0FWCc1JBXLUe+Wvbj/hWSw0csWs7VA/xWDCDEquxLTUzaxIGR+xgeg4qplbE0zmF0/JmVP6JEqMK41krv6emLDY2nEcus6Y4dAuezPxP6+TYXQdTKROMxSaLxZFmSKYkNn7pC+N4KjGjjBupLuV8CEzjKMLqeRCoMsvr5LmRZX6VXp/Wand5HEU4QRO4RwoXEEN7qAODeCg4Rle4c2z3ov37n0sWgtePnMMf+B9/gBHao/+AAAB8XicbVDLSgNBEOyNrxhfUY9eBoPgKeyIoMegF48RzAOTJcxOZpMhs7PLTK8QQv7CiwdFvPo33vwbJ8keNLGgoajqprsrTJW06PvfXmFtfWNzq7hd2tnd2z8oHx41bZIZLho8UYlph8wKJbVooEQl2qkRLA6VaIWj25nfehLGykQ/4DgVQcwGWkaSM3TSIyVdlLGwhPbKFb/qz0FWCc1JBXLUe+Wvbj/hWSw0csWs7VA/xWDCDEquxLTUzaxIGR+xgeg4qplbE0zmF0/JmVP6JEqMK41krv6emLDY2nEcus6Y4dAuezPxP6+TYXQdTKROMxSaLxZFmSKYkNn7pC+N4KjGjjBupLuV8CEzjKMLqeRCoMsvr5LmRZX6VXp/Wand5HEU4QRO4RwoXEEN7qAODeCg4Rle4c2z3ov37n0sWgtePnMMf+B9/gBHao/+


Pooling

3 ⇥ 3AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0msoMeiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8ATYyQAg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0msoMeiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8ATYyQAg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0msoMeiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8ATYyQAg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0msoMeiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8ATYyQAg==


Pooling

5 ⇥ 5AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AU66QBg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AU66QBg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AU66QBg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AU66QBg==


Pooling

Input Feature Maps

Output Feature Maps

Inception Block

Channel-Wise

AttentionInput

Feature Tensor

Spatial-Wise

AttentionOutput Feature Tensor

Attention Block

14 / 1

Our Solution: Inception Module

LmetricAAAB/XicdVDLSgMxFM3UV62v8bFzEyyCqzIjgl0W3bhwUcE+oB2GTHqnDU1mhiQj1GHwV9y4UMSt/+HOvzHTVvB5IORwzr3k5AQJZ0o7zrtVWlhcWl4pr1bW1jc2t+ztnbaKU0mhRWMey25AFHAWQUszzaGbSCAi4NAJxueF37kBqVgcXetJAp4gw4iFjBJtJN/e6wuiR5Tw7DL3MwFaMpr7dtWp1Z0C+Ddxa9PbqaI5mr791h/ENBUQacqJUj3XSbSXEakZ5ZBX+qmChNAxGULP0IgIUF42TZ/jQ6MMcBhLcyKNp+rXjYwIpSYiMJNFVvXTK8S/vF6qw7qXsShJNUR09lCYcqxjXFSBB0wC1XxiCKGSmayYjogkVJvCKqaEz5/i/0n7uOY6NffqpNo4m9dRRvvoAB0hF52iBrpATdRCFN2ie/SInqw768F6tl5moyVrvrOLvsF6/QB5MpXgAAAB/XicdVDLSgMxFM3UV62v8bFzEyyCqzIjgl0W3bhwUcE+oB2GTHqnDU1mhiQj1GHwV9y4UMSt/+HOvzHTVvB5IORwzr3k5AQJZ0o7zrtVWlhcWl4pr1bW1jc2t+ztnbaKU0mhRWMey25AFHAWQUszzaGbSCAi4NAJxueF37kBqVgcXetJAp4gw4iFjBJtJN/e6wuiR5Tw7DL3MwFaMpr7dtWp1Z0C+Ddxa9PbqaI5mr791h/ENBUQacqJUj3XSbSXEakZ5ZBX+qmChNAxGULP0IgIUF42TZ/jQ6MMcBhLcyKNp+rXjYwIpSYiMJNFVvXTK8S/vF6qw7qXsShJNUR09lCYcqxjXFSBB0wC1XxiCKGSmayYjogkVJvCKqaEz5/i/0n7uOY6NffqpNo4m9dRRvvoAB0hF52iBrpATdRCFN2ie/SInqw768F6tl5moyVrvrOLvsF6/QB5MpXgAAAB/XicdVDLSgMxFM3UV62v8bFzEyyCqzIjgl0W3bhwUcE+oB2GTHqnDU1mhiQj1GHwV9y4UMSt/+HOvzHTVvB5IORwzr3k5AQJZ0o7zrtVWlhcWl4pr1bW1jc2t+ztnbaKU0mhRWMey25AFHAWQUszzaGbSCAi4NAJxueF37kBqVgcXetJAp4gw4iFjBJtJN/e6wuiR5Tw7DL3MwFaMpr7dtWp1Z0C+Ddxa9PbqaI5mr791h/ENBUQacqJUj3XSbSXEakZ5ZBX+qmChNAxGULP0IgIUF42TZ/jQ6MMcBhLcyKNp+rXjYwIpSYiMJNFVvXTK8S/vF6qw7qXsShJNUR09lCYcqxjXFSBB0wC1XxiCKGSmayYjogkVJvCKqaEz5/i/0n7uOY6NffqpNo4m9dRRvvoAB0hF52iBrpATdRCFN2ie/SInqw768F6tl5moyVrvrOLvsF6/QB5MpXgAAAB/XicdVDLSgMxFM3UV62v8bFzEyyCqzIjgl0W3bhwUcE+oB2GTHqnDU1mhiQj1GHwV9y4UMSt/+HOvzHTVvB5IORwzr3k5AQJZ0o7zrtVWlhcWl4pr1bW1jc2t+ztnbaKU0mhRWMey25AFHAWQUszzaGbSCAi4NAJxueF37kBqVgcXetJAp4gw4iFjBJtJN/e6wuiR5Tw7DL3MwFaMpr7dtWp1Z0C+Ddxa9PbqaI5mr791h/ENBUQacqJUj3XSbSXEakZ5ZBX+qmChNAxGULP0IgIUF42TZ/jQ6MMcBhLcyKNp+rXjYwIpSYiMJNFVvXTK8S/vF6qw7qXsShJNUR09lCYcqxjXFSBB0wC1XxiCKGSmayYjogkVJvCKqaEz5/i/0n7uOY6NffqpNo4m9dRRvvoAB0hF52iBrpATdRCFN2ie/SInqw768F6tl5moyVrvrOLvsF6/QB5MpXg


Inputs

Backbone Network

Branch II: HotspotClassification

LpredictionAAACAXicdVDLSsNAFL2pr1pfUTeCm8EiuAqJCHZZdOPCRQX7gDaEyWTSDp08mJkIJcSNv+LGhSJu/Qt3/o2TtoLPC8MczrmHe+/xU86ksu13o7KwuLS8Ul2tra1vbG6Z2zsdmWSC0DZJeCJ6PpaUs5i2FVOc9lJBceRz2vXH56XevaFCsiS+VpOUuhEexixkBCtNeebeIMJqRDDPLwsv19aAkVIpPLNuWw27LPQbONb0t+swr5Znvg2ChGQRjRXhWMq+Y6fKzbFQjHBa1AaZpCkmYzykfQ1jHFHp5tMLCnSomQCFidAvVmjKfnXkOJJyEvm6s9xX/tRK8i+tn6mw4eYsTjNFYzIbFGYcqQSVcaCACUoUn2iAiWB6V0RGWGCidGg1HcLnpeh/0Dm2HNtyrk7qzbN5HFXYhwM4AgdOoQkX0II2ELiFe3iEJ+POeDCejZdZa8WYe3bhWxmvH7uzl7U=AAACAXicdVDLSsNAFL2pr1pfUTeCm8EiuAqJCHZZdOPCRQX7gDaEyWTSDp08mJkIJcSNv+LGhSJu/Qt3/o2TtoLPC8MczrmHe+/xU86ksu13o7KwuLS8Ul2tra1vbG6Z2zsdmWSC0DZJeCJ6PpaUs5i2FVOc9lJBceRz2vXH56XevaFCsiS+VpOUuhEexixkBCtNeebeIMJqRDDPLwsv19aAkVIpPLNuWw27LPQbONb0t+swr5Znvg2ChGQRjRXhWMq+Y6fKzbFQjHBa1AaZpCkmYzykfQ1jHFHp5tMLCnSomQCFidAvVmjKfnXkOJJyEvm6s9xX/tRK8i+tn6mw4eYsTjNFYzIbFGYcqQSVcaCACUoUn2iAiWB6V0RGWGCidGg1HcLnpeh/0Dm2HNtyrk7qzbN5HFXYhwM4AgdOoQkX0II2ELiFe3iEJ+POeDCejZdZa8WYe3bhWxmvH7uzl7U=AAACAXicdVDLSsNAFL2pr1pfUTeCm8EiuAqJCHZZdOPCRQX7gDaEyWTSDp08mJkIJcSNv+LGhSJu/Qt3/o2TtoLPC8MczrmHe+/xU86ksu13o7KwuLS8Ul2tra1vbG6Z2zsdmWSC0DZJeCJ6PpaUs5i2FVOc9lJBceRz2vXH56XevaFCsiS+VpOUuhEexixkBCtNeebeIMJqRDDPLwsv19aAkVIpPLNuWw27LPQbONb0t+swr5Znvg2ChGQRjRXhWMq+Y6fKzbFQjHBa1AaZpCkmYzykfQ1jHFHp5tMLCnSomQCFidAvVmjKfnXkOJJyEvm6s9xX/tRK8i+tn6mw4eYsTjNFYzIbFGYcqQSVcaCACUoUn2iAiWB6V0RGWGCidGg1HcLnpeh/0Dm2HNtyrk7qzbN5HFXYhwM4AgdOoQkX0II2ELiFe3iEJ+POeDCejZdZa8WYe3bhWxmvH7uzl7U=AAACAXicdVDLSsNAFL2pr1pfUTeCm8EiuAqJCHZZdOPCRQX7gDaEyWTSDp08mJkIJcSNv+LGhSJu/Qt3/o2TtoLPC8MczrmHe+/xU86ksu13o7KwuLS8Ul2tra1vbG6Z2zsdmWSC0DZJeCJ6PpaUs5i2FVOc9lJBceRz2vXH56XevaFCsiS+VpOUuhEexixkBCtNeebeIMJqRDDPLwsv19aAkVIpPLNuWw27LPQbONb0t+swr5Znvg2ChGQRjRXhWMq+Y6fKzbFQjHBa1AaZpCkmYzykfQ1jHFHp5tMLCnSomQCFidAvVmjKfnXkOJJyEvm6s9xX/tRK8i+tn6mw4eYsTjNFYzIbFGYcqQSVcaCACUoUn2iAiWB6V0RGWGCidGg1HcLnpeh/0Dm2HNtyrk7qzbN5HFXYhwM4AgdOoQkX0II2ELiFe3iEJ+POeDCejZdZa8WYe3bhWxmvH7uzl7U=

Attention Module

Concat

Convolution

Fully Connected

Inception Module

Pooling



Pooling

3 ⇥ 3AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0msoMeiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8ATYyQAg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0msoMeiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8ATYyQAg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0msoMeiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8ATYyQAg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0msoMeiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8ATYyQAg==


Pooling

5 ⇥ 5AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AU66QBg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AU66QBg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AU66QBg==AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Ae2oWy2m3bpZhN2J0IJ/RdePCji1X/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6wEnC/YgOlQgFo2ilxxrpoYi4IbV+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lipq1/jZ/OIpObPKgISxtqWQzNXfExmNjJlEge2MKI7MsjcT//O6KYbXfiZUkiJXbLEoTCXBmMzeJwOhOUM5sYQyLeythI2opgxtSCUbgrf88ippXVQ9t+rdX1bqN3kcRTiBUzgHD66gDnfQgCYwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AU66QBg==


Pooling

Input Feature Maps

Output Feature Maps

Inception Block

15 / 1

Our Solution: Channel-wise Attention Module



Inputs

Backbone Network



Attention Module

Concat

Convolution

Fully Connected

Channel-WiseAttention

Input Feature

Spatial-WiseAttention

Output Feature

Attention Block

Inception Module

W

H C(

AAAB8HicbVA9T8MwEL2Ur1K+CowsFi0SU5V0gbGChbFI9AM1UeW4TmrVdiLbQaqi/goWBhBi5eew8W9w2wzQ8qSTnt670929MOVMG9f9dkobm1vbO+Xdyt7+weFR9fikq5NMEdohCU9UP8SaciZpxzDDaT9VFIuQ0144uZ37vSeqNEvkg5mmNBA4lixiBBsrPdb9GxbHfl4fVmtuw10ArROvIDUo0B5Wv/xRQjJBpSEcaz3w3NQEOVaGEU5nFT/TNMVkgmM6sFRiQXWQLw6eoQurjFCUKFvSoIX6eyLHQuupCG2nwGasV725+J83yEx0HeRMppmhkiwXRRlHJkHz79GIKUoMn1qCiWL2VkTGWGFibEYVG4K3+vI66TYbntvw7pu1Vr2IowxncA6X4MEVtOAO2tABAgKe4RXeHOW8OO/Ox7K15BQzp/AHzucP02ePqA== AAAB8HicbVA9T8MwEL2Ur1K+CowsFi0SU5V0gbGChbFI9AM1UeW4TmrVdiLbQaqi/goWBhBi5eew8W9w2wzQ8qSTnt670929MOVMG9f9dkobm1vbO+Xdyt7+weFR9fikq5NMEdohCU9UP8SaciZpxzDDaT9VFIuQ0144uZ37vSeqNEvkg5mmNBA4lixiBBsrPdb9GxbHfl4fVmtuw10ArROvIDUo0B5Wv/xRQjJBpSEcaz3w3NQEOVaGEU5nFT/TNMVkgmM6sFRiQXWQLw6eoQurjFCUKFvSoIX6eyLHQuupCG2nwGasV725+J83yEx0HeRMppmhkiwXRRlHJkHz79GIKUoMn1qCiWL2VkTGWGFibEYVG4K3+vI66TYbntvw7pu1Vr2IowxncA6X4MEVtOAO2tABAgKe4RXeHOW8OO/Ox7K15BQzp/AHzucP02ePqA== AAAB8HicbVA9T8MwEL2Ur1K+CowsFi0SU5V0gbGChbFI9AM1UeW4TmrVdiLbQaqi/goWBhBi5eew8W9w2wzQ8qSTnt670929MOVMG9f9dkobm1vbO+Xdyt7+weFR9fikq5NMEdohCU9UP8SaciZpxzDDaT9VFIuQ0144uZ37vSeqNEvkg5mmNBA4lixiBBsrPdb9GxbHfl4fVmtuw10ArROvIDUo0B5Wv/xRQjJBpSEcaz3w3NQEOVaGEU5nFT/TNMVkgmM6sFRiQXWQLw6eoQurjFCUKFvSoIX6eyLHQuupCG2nwGasV725+J83yEx0HeRMppmhkiwXRRlHJkHz79GIKUoMn1qCiWL2VkTGWGFibEYVG4K3+vI66TYbntvw7pu1Vr2IowxncA6X4MEVtOAO2tABAgKe4RXeHOW8OO/Ox7K15BQzp/AHzucP02ePqA== AAAB8HicbVA9T8MwEL2Ur1K+CowsFi0SU5V0gbGChbFI9AM1UeW4TmrVdiLbQaqi/goWBhBi5eew8W9w2wzQ8qSTnt670929MOVMG9f9dkobm1vbO+Xdyt7+weFR9fikq5NMEdohCU9UP8SaciZpxzDDaT9VFIuQ0144uZ37vSeqNEvkg5mmNBA4lixiBBsrPdb9GxbHfl4fVmtuw10ArROvIDUo0B5Wv/xRQjJBpSEcaz3w3NQEOVaGEU5nFT/TNMVkgmM6sFRiQXWQLw6eoQurjFCUKFvSoIX6eyLHQuupCG2nwGasV725+J83yEx0HeRMppmhkiwXRRlHJkHz79GIKUoMn1qCiWL2VkTGWGFibEYVG4K3+vI66TYbntvw7pu1Vr2IowxncA6X4MEVtOAO2tABAgKe4RXeHOW8OO/Ox7K15BQzp/AHzucP02ePqA==

MaxPool

11 C

(


AvePool1

1 C(


11 C

(


11 C

(


11 C

(


W

H C(


Broadcast

InputFeature Maps

W

H C(


OutputFeature Maps

SharedEncoder-Decoder

11 C

(

AAAB8HicbVA9T8MwEL2Ur1K+CowsFi0SU5V0gbGChbFI9AM1UeW4TmrVdiLbQaqi/goWBhBi5eew8W9w2wzQ8qSTnt670929MOVMG9f9dkobm1vbO+Xdyt7+weFR9fikq5NMEdohCU9UP8SaciZpxzDDaT9VFIuQ0144uZ37vSeqNEvkg5mmNBA4lixiBBsrPdb9GxbHfl4fVmtuw10ArROvIDUo0B5Wv/xRQjJBpSEcaz3w3NQEOVaGEU5nFT/TNMVkgmM6sFRiQXWQLw6eoQurjFCUKFvSoIX6eyLHQuupCG2nwGasV725+J83yEx0HeRMppmhkiwXRRlHJkHz79GIKUoMn1qCiWL2VkTGWGFibEYVG4K3+vI66TYbntvw7pu1Vr2IowxncA6X4MEVtOAO2tABAgKe4RXeHOW8OO/Ox7K15BQzp/AHzucP02ePqA== AAAB8HicbVA9T8MwEL2Ur1K+CowsFi0SU5V0gbGChbFI9AM1UeW4TmrVdiLbQaqi/goWBhBi5eew8W9w2wzQ8qSTnt670929MOVMG9f9dkobm1vbO+Xdyt7+weFR9fikq5NMEdohCU9UP8SaciZpxzDDaT9VFIuQ0144uZ37vSeqNEvkg5mmNBA4lixiBBsrPdb9GxbHfl4fVmtuw10ArROvIDUo0B5Wv/xRQjJBpSEcaz3w3NQEOVaGEU5nFT/TNMVkgmM6sFRiQXWQLw6eoQurjFCUKFvSoIX6eyLHQuupCG2nwGasV725+J83yEx0HeRMppmhkiwXRRlHJkHz79GIKUoMn1qCiWL2VkTGWGFibEYVG4K3+vI66TYbntvw7pu1Vr2IowxncA6X4MEVtOAO2tABAgKe4RXeHOW8OO/Ox7K15BQzp/AHzucP02ePqA== AAAB8HicbVA9T8MwEL2Ur1K+CowsFi0SU5V0gbGChbFI9AM1UeW4TmrVdiLbQaqi/goWBhBi5eew8W9w2wzQ8qSTnt670929MOVMG9f9dkobm1vbO+Xdyt7+weFR9fikq5NMEdohCU9UP8SaciZpxzDDaT9VFIuQ0144uZ37vSeqNEvkg5mmNBA4lixiBBsrPdb9GxbHfl4fVmtuw10ArROvIDUo0B5Wv/xRQjJBpSEcaz3w3NQEOVaGEU5nFT/TNMVkgmM6sFRiQXWQLw6eoQurjFCUKFvSoIX6eyLHQuupCG2nwGasV725+J83yEx0HeRMppmhkiwXRRlHJkHz79GIKUoMn1qCiWL2VkTGWGFibEYVG4K3+vI66TYbntvw7pu1Vr2IowxncA6X4MEVtOAO2tABAgKe4RXeHOW8OO/Ox7K15BQzp/AHzucP02ePqA== AAAB8HicbVA9T8MwEL2Ur1K+CowsFi0SU5V0gbGChbFI9AM1UeW4TmrVdiLbQaqi/goWBhBi5eew8W9w2wzQ8qSTnt670929MOVMG9f9dkobm1vbO+Xdyt7+weFR9fikq5NMEdohCU9UP8SaciZpxzDDaT9VFIuQ0144uZ37vSeqNEvkg5mmNBA4lixiBBsrPdb9GxbHfl4fVmtuw10ArROvIDUo0B5Wv/xRQjJBpSEcaz3w3NQEOVaGEU5nFT/TNMVkgmM6sFRiQXWQLw6eoQurjFCUKFvSoIX6eyLHQuupCG2nwGasV725+J83yEx0HeRMppmhkiwXRRlHJkHz79GIKUoMn1qCiWL2VkTGWGFibEYVG4K3+vI66TYbntvw7pu1Vr2IowxncA6X4MEVtOAO2tABAgKe4RXeHOW8OO/Ox7K15BQzp/AHzucP02ePqA== 11

C/4(


11 C

(


Shared Encoder-Decoder

�()AAAB/nicbVBLSwMxGMz6rPW1Kp68BItQL2VXBD0WvXisYB/QXUo2m21D81iSrFCWgn/FiwdFvPo7vPlvzLZ70NaBkGHm+8hkopRRbTzv21lZXVvf2KxsVbd3dvf23YPDjpaZwqSNJZOqFyFNGBWkbahhpJcqgnjESDca3xZ+95EoTaV4MJOUhBwNBU0oRsZKA/c4iCSL9YTbKw80HXJUP58O3JrX8GaAy8QvSQ2UaA3cryCWOONEGMyQ1n3fS02YI2UoZmRaDTJNUoTHaEj6lgrEiQ7zWfwpPLNKDBOp7BEGztTfGzniukhoJzkyI73oFeJ/Xj8zyXWYU5Fmhgg8fyjJGDQSFl3AmCqCDZtYgrCiNivEI6QQNraxqi3BX/zyMulcNHyv4d9f1po3ZR0VcAJOQR344Ao0wR1ogTbAIAfP4BW8OU/Oi/PufMxHV5xy5wj8gfP5A3HylcY=AAAB/nicbVBLSwMxGMz6rPW1Kp68BItQL2VXBD0WvXisYB/QXUo2m21D81iSrFCWgn/FiwdFvPo7vPlvzLZ70NaBkGHm+8hkopRRbTzv21lZXVvf2KxsVbd3dvf23YPDjpaZwqSNJZOqFyFNGBWkbahhpJcqgnjESDca3xZ+95EoTaV4MJOUhBwNBU0oRsZKA/c4iCSL9YTbKw80HXJUP58O3JrX8GaAy8QvSQ2UaA3cryCWOONEGMyQ1n3fS02YI2UoZmRaDTJNUoTHaEj6lgrEiQ7zWfwpPLNKDBOp7BEGztTfGzniukhoJzkyI73oFeJ/Xj8zyXWYU5Fmhgg8fyjJGDQSFl3AmCqCDZtYgrCiNivEI6QQNraxqi3BX/zyMulcNHyv4d9f1po3ZR0VcAJOQR344Ao0wR1ogTbAIAfP4BW8OU/Oi/PufMxHV5xy5wj8gfP5A3HylcY=AAAB/nicbVBLSwMxGMz6rPW1Kp68BItQL2VXBD0WvXisYB/QXUo2m21D81iSrFCWgn/FiwdFvPo7vPlvzLZ70NaBkGHm+8hkopRRbTzv21lZXVvf2KxsVbd3dvf23YPDjpaZwqSNJZOqFyFNGBWkbahhpJcqgnjESDca3xZ+95EoTaV4MJOUhBwNBU0oRsZKA/c4iCSL9YTbKw80HXJUP58O3JrX8GaAy8QvSQ2UaA3cryCWOONEGMyQ1n3fS02YI2UoZmRaDTJNUoTHaEj6lgrEiQ7zWfwpPLNKDBOp7BEGztTfGzniukhoJzkyI73oFeJ/Xj8zyXWYU5Fmhgg8fyjJGDQSFl3AmCqCDZtYgrCiNivEI6QQNraxqi3BX/zyMulcNHyv4d9f1po3ZR0VcAJOQR344Ao0wR1ogTbAIAfP4BW8OU/Oi/PufMxHV5xy5wj8gfP5A3HylcY=AAAB/nicbVBLSwMxGMz6rPW1Kp68BItQL2VXBD0WvXisYB/QXUo2m21D81iSrFCWgn/FiwdFvPo7vPlvzLZ70NaBkGHm+8hkopRRbTzv21lZXVvf2KxsVbd3dvf23YPDjpaZwqSNJZOqFyFNGBWkbahhpJcqgnjESDca3xZ+95EoTaV4MJOUhBwNBU0oRsZKA/c4iCSL9YTbKw80HXJUP58O3JrX8GaAy8QvSQ2UaA3cryCWOONEGMyQ1n3fS02YI2UoZmRaDTJNUoTHaEj6lgrEiQ7zWfwpPLNKDBOp7BEGztTfGzniukhoJzkyI73oFeJ/Xj8zyXWYU5Fmhgg8fyjJGDQSFl3AmCqCDZtYgrCiNivEI6QQNraxqi3BX/zyMulcNHyv4d9f1po3ZR0VcAJOQR344Ao0wR1ogTbAIAfP4BW8OU/Oi/PufMxHV5xy5wj8gfP5A3HylcY=

~T ′ = Ac(~T)⊗ ~T, (1)Ac(~T) = σ(ED(AvgPool(~T)) + ED(MaxPool(~T))). (2)

16 / 1

Our Solution: Spatial-wise Attention Module



Inputs

Backbone Network



Attention Module

Concat

Convolution

Fully Connected

Channel-WiseAttention

Input Feature

Spatial-WiseAttention

Output Feature

Attention Block

Inception Module

Input Feature Maps

W

H C(


Cat(Max-Pooled, AvgPooled)W

(2

H

Convolution Broadcast

W

H C(


HW

(


C

Output Feature Maps

�()AAAB/nicbVBLSwMxGMz6rPW1Kp68BItQL2VXBD0WvXisYB/QXUo2m21D81iSrFCWgn/FiwdFvPo7vPlvzLZ70NaBkGHm+8hkopRRbTzv21lZXVvf2KxsVbd3dvf23YPDjpaZwqSNJZOqFyFNGBWkbahhpJcqgnjESDca3xZ+95EoTaV4MJOUhBwNBU0oRsZKA/c4iCSL9YTbKw80HXJUP58O3JrX8GaAy8QvSQ2UaA3cryCWOONEGMyQ1n3fS02YI2UoZmRaDTJNUoTHaEj6lgrEiQ7zWfwpPLNKDBOp7BEGztTfGzniukhoJzkyI73oFeJ/Xj8zyXWYU5Fmhgg8fyjJGDQSFl3AmCqCDZtYgrCiNivEI6QQNraxqi3BX/zyMulcNHyv4d9f1po3ZR0VcAJOQR344Ao0wR1ogTbAIAfP4BW8OU/Oi/PufMxHV5xy5wj8gfP5A3HylcY=AAAB/nicbVBLSwMxGMz6rPW1Kp68BItQL2VXBD0WvXisYB/QXUo2m21D81iSrFCWgn/FiwdFvPo7vPlvzLZ70NaBkGHm+8hkopRRbTzv21lZXVvf2KxsVbd3dvf23YPDjpaZwqSNJZOqFyFNGBWkbahhpJcqgnjESDca3xZ+95EoTaV4MJOUhBwNBU0oRsZKA/c4iCSL9YTbKw80HXJUP58O3JrX8GaAy8QvSQ2UaA3cryCWOONEGMyQ1n3fS02YI2UoZmRaDTJNUoTHaEj6lgrEiQ7zWfwpPLNKDBOp7BEGztTfGzniukhoJzkyI73oFeJ/Xj8zyXWYU5Fmhgg8fyjJGDQSFl3AmCqCDZtYgrCiNivEI6QQNraxqi3BX/zyMulcNHyv4d9f1po3ZR0VcAJOQR344Ao0wR1ogTbAIAfP4BW8OU/Oi/PufMxHV5xy5wj8gfP5A3HylcY=AAAB/nicbVBLSwMxGMz6rPW1Kp68BItQL2VXBD0WvXisYB/QXUo2m21D81iSrFCWgn/FiwdFvPo7vPlvzLZ70NaBkGHm+8hkopRRbTzv21lZXVvf2KxsVbd3dvf23YPDjpaZwqSNJZOqFyFNGBWkbahhpJcqgnjESDca3xZ+95EoTaV4MJOUhBwNBU0oRsZKA/c4iCSL9YTbKw80HXJUP58O3JrX8GaAy8QvSQ2UaA3cryCWOONEGMyQ1n3fS02YI2UoZmRaDTJNUoTHaEj6lgrEiQ7zWfwpPLNKDBOp7BEGztTfGzniukhoJzkyI73oFeJ/Xj8zyXWYU5Fmhgg8fyjJGDQSFl3AmCqCDZtYgrCiNivEI6QQNraxqi3BX/zyMulcNHyv4d9f1po3ZR0VcAJOQR344Ao0wR1ogTbAIAfP4BW8OU/Oi/PufMxHV5xy5wj8gfP5A3HylcY=AAAB/nicbVBLSwMxGMz6rPW1Kp68BItQL2VXBD0WvXisYB/QXUo2m21D81iSrFCWgn/FiwdFvPo7vPlvzLZ70NaBkGHm+8hkopRRbTzv21lZXVvf2KxsVbd3dvf23YPDjpaZwqSNJZOqFyFNGBWkbahhpJcqgnjESDca3xZ+95EoTaV4MJOUhBwNBU0oRsZKA/c4iCSL9YTbKw80HXJUP58O3JrX8GaAy8QvSQ2UaA3cryCWOONEGMyQ1n3fS02YI2UoZmRaDTJNUoTHaEj6lgrEiQ7zWfwpPLNKDBOp7BEGztTfGzniukhoJzkyI73oFeJ/Xj8zyXWYU5Fmhgg8fyjJGDQSF

Documents

Hotspot Detection via Attention-based Deep Layout Metric Learningbyu/papers/C106-ICCAD2020-Metric... · 2020. 10. 29. · Hotspot Detection via Attention-based Deep Layout Metric