Basics Image Processing, Data Handling, and Storage

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Basics Image Processing,Data Handling, and Storage

David Miguel Susano Pinto

Micron Advanced Microscopy Course, 2020

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

What is an image?

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

What is an image?Pixel data Metadata

• emission wavelength 535nm• excitation wavelength 500nm• exposure time 2s• objective Olympus 100X/1.40• pixel size 64.4×64.4 nm

• deconvolved• 8bit conversion after contrast adjustment• full range displayed

• horse fibroblast cell line• transient transfection of H2B-GFP• treated with sugar lumps

File ⇒ my-lovely-horse-H2B-GFP 01 R3D D3D.dv

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Pixel data

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Camera

Photons ⇒ Charge ⇒ Digital number

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Camera

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Camera

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Histogram of pixel values

0 10000 20000 30000 40000 50000 600000

2000

4000

6000

8000

10000

Adjust contrast/levels, see effect on display and pixel values.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Bit depth / dynamic rangenbits1 2 2

2 2× 2 43 2× 2× 2 84 24 165 25 326 26 647 27 1288 28 2569 29 51210 210 102411 211 2048

. . .15 215 3276816 216 65536

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Bit depth / dynamic rangenbits1 2 22 2× 2 4

3 2× 2× 2 84 24 165 25 326 26 647 27 1288 28 2569 29 51210 210 102411 211 2048

. . .15 215 3276816 216 65536

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Bit depth / dynamic rangenbits1 2 22 2× 2 43 2× 2× 2 8

4 24 165 25 326 26 647 27 1288 28 2569 29 51210 210 102411 211 2048

. . .15 215 3276816 216 65536

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Bit depth / dynamic rangenbits1 2 22 2× 2 43 2× 2× 2 84 24 16

5 25 326 26 647 27 1288 28 2569 29 51210 210 102411 211 2048

. . .15 215 3276816 216 65536

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Bit depth / dynamic rangenbits1 2 22 2× 2 43 2× 2× 2 84 24 165 25 32

6 26 647 27 1288 28 2569 29 51210 210 102411 211 2048

. . .15 215 3276816 216 65536

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Bit depth / dynamic rangenbits1 2 22 2× 2 43 2× 2× 2 84 24 165 25 326 26 647 27 1288 28 256

9 29 51210 210 102411 211 2048

. . .15 215 3276816 216 65536

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Bit depth / dynamic rangenbits1 2 22 2× 2 43 2× 2× 2 84 24 165 25 326 26 647 27 1288 28 2569 29 51210 210 102411 211 2048

. . .15 215 3276816 216 65536

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Pixel data

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Images as Signals

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Images as Signals

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Images as Signals

Km

Vmax

½Vmax

0 1000 2000 3000 40000.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Substrate concentration

Rea

ctio

n ra

te

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Images as Signals

Serra da Freita

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Images as Signals

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Images as Signals

0 100 200 300 400 5002000

4000

6000

8000

10000

12000

14000

0 100 200 300 400 5002000

4000

6000

8000

10000

12000

14000

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Images as Signals

0 100 200 300 400 5002000

4000

6000

8000

10000

12000

14000

0 100 200 300 400 5002000

4000

6000

8000

10000

12000

14000

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Images as Surfaces

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Images as ND Arrays

• x and y• time• z (volume)• wavelength• phase• stage angle

Think ”data”, not ”picture”

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

RGB

If you ever get an RGB image, you did something wrong.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

RGB

If you ever get an RGB image, you did something wrong.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

multi channel

Multichannel image is a series of grayscale images.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Lookup tables or colormaps

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

color index

0 50 100 150 200 250color index

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

color index

0 50 100 150 200 250color index

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

color index

0 50 100 150 200 250color index

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

color index

0 50 100 150 200 250color index

See ImageJ “Display LUTs”

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Lookup tables or colormaps

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

color index

0 50 100 150 200 250color index

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

color index

0 50 100 150 200 250color index

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

color index

0 50 100 150 200 250color index

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

color index

0 50 100 150 200 250color index

See ImageJ “Display LUTs”

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Lightness perception

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Qualitative or Categorical

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Qualitative or Categorical

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Qualitative or Categorical

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Direct record

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Image Reconstruction

• Localisation Microscopy (STORM/PALM)• Structured Illumination

Two step process

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

raster vs vector

raster individual pixel values. Use ImageJ.vector polygons with attributes. Use inkscape.

ImageJ Inkscape GIMP

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Figure preparation guidelines

• Carry out all processing and analysis of images beforemaking figures by using pixel based (raster) programs.• Handling of images for figures should use pixel based

(raster) programs.• Vector graphics for lettering, arrows, diagrams, arranging

panels.• Both can rotate, resize, and crop.• Do not use office applications like powerpoint, keynote,

writer, word, or impress.• Do not use screenshoot.• Be consistent with processing steps, especially contrasting.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Compression

Save TIFF, not JPEG.

A useful generalisation.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Compression

Save TIFF, not JPEG.A useful generalisation.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Image data is large

1024 × 1024, 20 Z slices = 20MB

Most cameras are 16bit, that’s 40MBDo it in two channels, that’s 80MB

Every minute for 1 hour, you now have a 5GB fileIf you visit 10 points, that’s 50GB worth of data.

You have just reached Zenodo’s limit.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Image data is large

1024 × 1024, 20 Z slices = 20MBMost cameras are 16bit, that’s 40MB

Do it in two channels, that’s 80MBEvery minute for 1 hour, you now have a 5GB fileIf you visit 10 points, that’s 50GB worth of data.

You have just reached Zenodo’s limit.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Image data is large

1024 × 1024, 20 Z slices = 20MBMost cameras are 16bit, that’s 40MBDo it in two channels, that’s 80MB

Every minute for 1 hour, you now have a 5GB fileIf you visit 10 points, that’s 50GB worth of data.

You have just reached Zenodo’s limit.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Image data is large

1024 × 1024, 20 Z slices = 20MBMost cameras are 16bit, that’s 40MBDo it in two channels, that’s 80MB

Every minute for 1 hour, you now have a 5GB file

If you visit 10 points, that’s 50GB worth of data.You have just reached Zenodo’s limit.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Image data is large

1024 × 1024, 20 Z slices = 20MBMost cameras are 16bit, that’s 40MBDo it in two channels, that’s 80MB

Every minute for 1 hour, you now have a 5GB fileIf you visit 10 points, that’s 50GB worth of data.

You have just reached Zenodo’s limit.

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

where

1 you do it2 someone else does it

1 one disk per person2 shared filesystem3 OMERO

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

where

1 you do it2 someone else does it

1 one disk per person2 shared filesystem3 OMERO

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Fileset

• one file one image

• one file many images• many files one image

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Fileset

• one file one image• one file many images

• many files one image

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Fileset

• one file one image• one file many images• many files one image

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Files and metadata

• why some many formats?• metadata saved may be format dependent• use libraries and applications that respect you• open formats

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Bioformats and OME-TIFF

What?Image?

Camera

Histograms

Bitdepth

Images as ND Arrays

Colour

LUT

LUT

ImageReconstruction

Figuresraster vs vector

guidelines

Compression

StorageOMERO

Image Files

OME

Open Microscopy Environment

• bioformats• OME-XML• OME-TIFF• OMERO• IDR