STRUCT @ PKU Deep Joint Rain Detection and Removal from a ... · IEEE 2017 Conference on Computer...

IEEE 2017 Conference on

Computer Vision and Pattern

Recognition

Introduction

Motivation

Recurrent Joint Rain Detection and Removal Experiment Results

Deep Joint Rain Detection and Removal from a Single ImageWenhan Yang1, Robby T. Tan2,3, Jiashi Feng2, Jiaying Liu1, Zongming Guo1, and Shuicheng Yan4,2

1Institute of Computer Science and Technology, Peking University, Beijing, China2National University of Singapore, 3Yale-NUS College, 4360 AI Institute

IEEE 2017 Conference on Computer

Vision and Pattern Recognition

Contextualized Dilated Network

Visual comparison

Hard case

- The intrinsic overlapping between

rain streaks and texture.

- Complex degradations, e.g. heavy

rain and mist.

- A limited receptive field.

Challenges

- Region-dependent rain models

Rain-streak binary mask

Diversified streaks & accumulation

- Joint rain detection and removal

- Contextualized dilated network

Contributions

Rain Image Formation

1

1s

t

t

O = B + S R A

Region dependent

- Rain localization.

- Allowing a new rain removal pipeline to detect rain regions first,

and then to operate differently.

Mist and heavy rains

Motivation

- A larger receptive field.

- More context information.

ResNet

- Increasingly larger receptive

fields.

Multi-Path Dilation

Convolutions

- Filter at different scales

- P1: 5×5

- P2: 9×9

- P3: 13×13

Rain detection, estimation and removal

F R, F,R S, , ,

F,R S O RS B

Rain Images DSC[1] LP[2] Ours

For more details & codes, scan QR code or navigate

to http://www.icst.pku.edu.cn/struct/Projects/joint_rain_removal.html

Interested in our team STRUCT? Navigate to

1. Y. Li, et al. Rain streak removal using layer

priors. CVPR, 2016.

2. Y. Luo, et al. Removing rain from a single

image via discriminative sparse coding.

ICCV, 2015.

STRUCT @ PKUSpatial and Temporal Restoration, Understanding and Compression

YaleNUSCollege

NUSNational Universityof Singapore

http://www.icst.pku.edu.cn/struct/struct.html