Computer Vision Established Methods and their Adaptation ......© GEOMAR, Anne Jordt, Kevin Köser, 2015 Computer Vision – Established Methods and their Adaptation to Underwater

© GEOMAR, Anne Jordt, Kevin Köser, 2015


Computer Vision – Established Methods and

their Adaptation to Underwater Imaging

Anne Jordt

August 19th, 2015

Rostock, Germany

1 DeepSea Monitoring, GEOMAR Helmholtz

Centre for Ocean Research 2 Multimedia Information Processing Group, Kiel

University

1,2


Helmholtz Centre for Ocean Research Kiel • Foundation by public law • Member of the Helmholtz Association of German Research

Centers • Budget: 60 Mio. Euro, about 30 mio. institutional funding, 30

mio. project funding • Staff: 750, about 400 scientists • Close relationship to Kiel University Kiel: joint professorships,

curricula, large research projects: cluster of excellence „The future ocean“, collaborative research projects 574 & 754

GEOMAR

Marine Research with tradition and innovation

Dr. Andreas Villwock


Major research topics From the deep-sea to the atmosphere

Helmholtz Centre for Ocean Research Kiel

Dr. Andreas Villwock

• The role of the ocean in climate change: temperature and sea level rise, extreme events, CO2 budget

• Anthropogenic impact on marine ecosystems: food webs, ecosystem and climate change, ocean acidification, overfishing, aliens

• Marine resources: natural substances from the sea, gas hydrates, mineral resources

• Plate tectonics and natural hazards: subduction zones volcanism, and tsunamis


4

- Introduction

- Features and Feature Matching

- Geometry of Image Formation

- Calibration

- Structure from Motion

- Dense Stereo

- Conclusion

Outline


Computer Vision – 3D Reconstruction

5

Bundler reconstructs 3D information from www.flickr.com large image sets (image from

http://www.cs.cornell.edu/~snavely/bundler/).

Introduction

Snavely et al. 2007

Example images bundler have been removed in this version

due to copy right reasons.

Please refer to the link below.


Computer Vision – Motion Capture

6

Motion capturing for movies and computer games (image from

http://www.businessinsider.com/benedict-cumberbatch-motion-capture-smaug-hobbit-

2014-10?IRT)

Introduction

Example image for motion capture has been

removed in this version due to copy right reasons.

Please refer to the article linked below.


Computer Vision – Autonomous Driving

7

Retrieving 3D information of the scene in front of a car for autonomous driving (image

from Badino et al. 2009).

Introduction

Badino et al. 2009

Example image of SGM and Daimler’s stixel world has been


Please refer to the paper.


Computer Vision – Autonomous Driving

8

Scene understanding through segmentation and tracking (image from http://www.6d-

vision.com/scene-labeling).

Introduction

Scharwächter et al. 2013

Example image of scene understanding has been removed in

this version due to copy right reasons.

Please refer to the webpage.


Computer Vision – Mars Rover

9

Photo mosaicking on Mars (image from

http://www.nasa.gov/mission_pages/msl/images/index.html)

Introduction

Example images of the Mars Rover and one of the photo

mosaiks have been removed in this version due to copy right

reasons.



Computer Vision – Face Detection

10

Automatic face detection in images (images from

http://cs.brown.edu/courses/cs143/2011/results/proj4/psastras/)

Introduction

Example images of face recognition have been removed in this

version due to copy right reasons.



Effects of Water on Image Formation

11

• light traveling through

water is attenuated and

scattered -> effects on

image color

• light entering the

underwater housing is

refracted -> effects on

image geometry

Introduction


Scientific Applications - Geology

12

Left: underwater volcano, near Cape Verdes, water depth ca. 3500m (ROV team,

GEOMAR). Right: hydrothermal vent, Middle Atlantic Ridge (ROV team, GEOMAR).

Introduction


Scientific Applications - Archaeology

13

Left: Hedvig Sophia shipwreck (Florian Huber, Kiel University). Right: human skull

found in underwater cave system in Yucatan, Mexico (Christian Howe, Kiel University).

Introduction


Scientific Applications – Ocean and Atmosphere

14

Natural methane release (North Sea) and lab gas experiments

Introduction


Scientific Applications – Habitat Mapping and Monitoring

15

Left: cold water corals (Jago team, GEOMAR).

Introduction


Scientific Applications - Biology

16

Left: sloth skeleton in cave system in Yucatan, Mexico (Uli Kunz, Kiel University).

Right: shrimp (GEOMAR).

Introduction


Other Applications

17

- construction, e.g. harbor and bridge construction, oil rig maintenance

- deep sea mining, e.g. manganese nodules, massive sulfides

Left: Manganese nodules on seafloor (GEOMAR). Right: Manganese nodule

(GEOMAR).

Introduction


Capturing Underwater Images - Vehicles

18

• Remotely Operated Vehicle (ROV)

• tethered to the ship usually with real time video

data transmission

• equipped with thrusters (steered from pilot on

the ship)

• usually provides lighting

Top: ROV Kiel 6000 (GEOMAR, depth 6000 m)

Bottom: OpenROV, self-made, low cost ROV that

can be ordered by anyone and needs assembly

(water depth max. 50 m) from: www.openrov.org.

Introduction


Capturing Underwater Images - Vehicles

19

• Autonomous Underwater Vehicle (AUV)

• can navigate on pre-defined path autonomously

• equipped with thruster and different hardware for taking samples, not

necessarily cameras

• if equipped with cameras often strobe lights

• limited power supply

Image: AUV Abyss (GEOMAR). Equipped with sonar systems and digital stills

camera. Length: 4 m, depth rating 6000 m

Introduction


Capturing Underwater Images - Divers

20

• require extensive training

• equipped with cameras and lights

• limited diving time and depth

• humans have far better

capabilities of reacting to

environment

Image: scientific diver from Kiel

University with camera, light, and

scooter (Florian Huber).

Introduction


Capturing Underwater Images - Lighting

21

Lighting underwater scenes:

- sunlight in water depths close to surface

- in deep water lighting from ROVs and AUVs

- power consumption

Introduction


Capturing Underwater Images - Cameras

22

Different camera systems in pressure

housings:

- consumer hand-helds

- cameras used by divers

- cameras used on ROVs and AUVs

- GoPros

- OpenROV with cylinder housing

Cylindrical from:

www.openrov.org Dome port

Flat port

Introduction


Nuisances

23

Left: floating particles. Right: sunlight caustics caused by surface waves (images from

http://webee.technion.ac.il/people/yoav/research/flicker.html).

Swirski et al. 2007

Introduction

Example images of marine snow and flickering caustics have been




Challenges

24

• difficult imaging situations i.e. danger of water leakage, water pressure, salinity,

weather, difficult data transfer, bad visibility

• technical effort and complexity are MUCH higher than in air, especially compared to

lab environments

• images often captured on scientific cruises by people from other disciplines

• expensive; images cannot be taken again; often something missing (e.g. calibration)

• special problems with manual steps, but huge amounts of data, and growing

Introduction


References

25

R. Szeliski, Computer Vision Algorithms and Applications, Springer 2011.

N. Snavely, S. M. Seitz, R. Szeliski. Modeling the World from Internet Photo

Collections. International Journal of Computer Vision, 2007.

H. Badino, U. Franke, D. Pfeiffer. The stixel world – a compact medium level

representation of the 3d-world. Pattern Recognition, 2009

T. Scharwächter, M. Enzweiler, S. Roth, and U. Franke. Efficient Multi-Cue Scene

Segmentation, In Proc. of the German Conference on Pattern Recognition

(GCPR), 2013

Y. Swirski, Y. Y. Schechner, B. Herzberg, and S. Negahdaripour, Stereo from

flickering caustics, Proc. IEEE ICCV (2009).

Introduction


Wrap up

26

• computer vision has the goal of automated image understanding

• multiple interesting applications for underwater vision

• different cameras, vehicles, and lighting situations to be taken into account

• two major effects on underwater image formation

• scattering and attenuation effects => appearance

• refraction at underwater housing => geometry

Introduction

Documents

Computer Vision Established Methods and their Adaptation ......© GEOMAR, Anne Jordt, Kevin Köser, 2015 Computer Vision – Established Methods and their Adaptation to Underwater