-the Computational Image Analysis role

What are our digital images trying to tell us?- the Computational Image Analysis role

Dr Andrew SquelchCurtin Institute for Computation

Department of Exploration Geophysics

http://www.itqb.unl.pt/labs/bacterial-cell-biology

Cell biology –various microscopy sources e.g. FM, SIM

If only images could talk…

https://www.quora.com/What-is-the-difference-between-tablets-and-pills

Manufactured particles/objects –various image sources

https://www.zeiss.com/content/dam/Microscopy/Segments/Raw%20Materials/Oil%20and%20Gas/Rock%20Characterization/sandstone.jpg

Mineral particles/rock grains –Microscope / FIB-SEM/ microCT 2D & 3D

https://au.pinterest.com/toddsangels1/chapter-5-histology/

Histology/pathology –various imaging methods, e.g. SHG

If only images could talk…

Human interest –Digital photograph

http://www.channelinstruments.com/v_1/1249.aspx http://www.channelinstruments.com/v_1/1249.aspx

Canopy cover –Digital photograph (E&A)

http://www.stri.si.edu/sites/esp/tesp/plant_pictures/i_sp1391.mx.jpg

Plant / crop health –Scanned digital image

http://www.pnas.org/content/102/21/7760/F1.large.jpg

Motion / growth rates –Time series images

Seminar objectives

• Raise awareness about CIA

• Offer helping hand to new users of CIA

• Connect with researchers that have common interest in (some) aspects of CIA

• Canvas interest in CIA Research Group / SIG

• Cross-disciplinary CIA projects, e.g.

– Extractive metallurgy + Geophysics + Computer Science

Computational Image Analysis

“Seeing is believing : Quantifying is convincing”

“Image analysis in biology is moving from seeing and observing to quantifying

and modeling. Interpreting images as scientific measurements, rather than as

mere visualizations, brings the need for uncertainty quantification, error analysis,

statistical inference frameworks, etc. This raises a number of exciting theoretical

and algorithmic questions.”

(Sbalzarini, 2016)

Sbalzarini, I. F. 2016. Seeing is believing - quantifying is convincing: Computational image analysis in biology. W.H. De Vos et al. (eds.), Focus on Bio-Image Informatics, Advances in Anatomy, Embryology and Cell Biology 219. Springer International Publishing, Switzerland.

Computational Image Analysis

• Computer-assisted or automated analysis of digital images

• Repeatability cf manual annotation and analysis

• Batch processing large sets of digital images (inc. HPC)

• 2D, 3D and tracking in multi-channel and time-series (4D and 5D)

• Borrows from image processing, computer vision and machine learning

• Quantifying feature attributes in digital images– e.g. shape factors, size, orientation, waviness, growth, motion…

• Not always correct – need calibration/validation, error assessment

Open Source CIA software & toolkits

Fiji / ImageJ / ImageJ2https://fiji.sc/ https://imagej.net/

Icyhttp://icy.bioimageanalysis.org/

CellProfilerhttp://cellprofiler.org/

Blackspot (Python, PIL, Numpy, SciPy)https://www.ncbs.res.in/blackspot

Python (PIL, Numpy, SciPy, Scikit-image, OpenCV)https://www.python.org/

R Statistical computing languagehttps://www.r-project.org/

KNIME Analytics Platform (machine learning)https://www.knime.org/knime-analytics-platform

Many more plant analysis examples here: http://www.plant-image-analysis.org

VTK, ITK and 3D slicerhttps://www.kitware.com

Drishtihttps://github.com/nci/drishti

http://www.plant-image-analysis.org/

Commercial CIA software & toolkits

Amira (Life science) & Avizo (Materials Science)

https://www.fei.com/software/amira-avizo/

Imaris

http://www.bitplane.com/imaris/imarisMIPAR

http://www.mipar.us

Mimics

http://www.materialise.com/en/medical/software/mimics

ScanIP

https://simpleware.com/software/scanip/

MATLAB

https://au.mathworks.com/

What is Image Analysis / Quantification?

https://imagej.github.io/presentations/fiji-introduction/#/15/8

Are all clown noses the same size?

ImageJ Sample images


https://au.pinterest.com/pin/445574956854087709/


https://au.pinterest.com/pin/445574956854087709/

CIA Workflow elements

• Prepare, Identify, Analyse



Measuring & Analysis

• Histogram

• Shape attributes

• Orientation

• Distribution

• Coherency

• Quantification

• Visualisation

Segmentation & Labelling

• Histogram

• Thresholding

• Edge detection

• Morphology operations

• Binarisation

• Fill holes

• Create mask

• Watershed

• Identify

• Labelling

• ROIs

• Tracking

Image processing

• Contrast equalization

• Background removal

• Filtering

• Denoising

• Sharpening

• Edge preserving

• Smoothing

• Registration

• Alignment




• Histogram


• Orientation

• Distribution

• Coherency

• Quantification

• Visualisation


• Histogram

• Thresholding

• Edge detection


• Binarisation

• Fill holes

• Create mask

• Watershed

• Identify

• Labelling

• ROIs

• Tracking

Image processing



• Filtering

• Denoising

• Sharpening

• Edge preserving

• Smoothing

• Registration

• Alignment




• Histogram


• Orientation

• Distribution

• Coherency

• Quantification

• Visualisation


• Histogram

• Thresholding

• Edge detection


• Binarisation

• Fill holes

• Create mask

• Watershed

• Identify

• Labelling

• ROIs

• Tracking

Image processing



• Filtering

• Denoising

• Sharpening

• Edge preserving

• Smoothing

• Registration

• Alignment

Workflow example

http://www.albany.edu/celltracking/

CIA examples from a few discipline areas

Plant leaf area - Blackspot

https://www.ncbs.res.in/sites/default/files/Black_Spot_User_Manual.pdf

Leaf area is calculated in square centimetres (cm2) using the number of ‘leaf’ pixels in the binary image and pixel resolution values

extracted from the image exif data

Canopy cover - Canopy Analyser

Automated Analysis of in Situ Canopy Images for the Estimation of Forest Canopy CoverKorhonen, L. and Heikkinen, J. 2009. Forest Science, Volume 55 (4), 323-334

Materials CT analysis -Avizo

https://en.wikipedia.org/wiki/Avizo_(software)

Rock porosity and permeability at scale

Threshold applied to segment microCT image (left) into binary image (right)

Liu, J., Regenauer-Lieb, K., Hines, C., Liu, K., Gaede, O. and Squelch, A. Improved estimates of percolation and anisotropic permeability from 3-D X-ray microtomography using stochastic analyses and visualization, Geochem. Geophys. Geosyst., 10, Q05010, doi:10.1029/2008GC002358. 2009.


Gray scale histogram –separates pores from grains



Comparison of tensors of permeability and pore clusters

Local porosity distribution analysed for different sub-volumes: >800µm the value stabilises


Fibre analysis - DiameterJ

https://imagej.net/DiameterJ

(Top) SEM image --> Segmented image --> Stylized Euclidean distance transform. (Bottom) A few of the graphs capable of being produced from data given by DiameterJ.

Nuclei size attributes – LAS X

http://www.leica-microsystems.com/products/microscope-software/details/product/leica-las-x-ls/

CellProfiler

http://cellprofiler.org/manuals/current/

Nuclei shape factor analysis

Choi, S., Wang, W., Ribeiro, A. J. S., Kalinowski, A., Gregg, S. Q., Opresko, P. L., … Dahl, K. N. (2011). Computational image analysis of nuclear morphology associated with various nuclear-specific aging disorders. Nucleus, 2(6), 570–579. http://doi.org/10.4161/nucl.2.6.17798

Nuclei shape factor analysis

Choi, S., Wang, W., Ribeiro, A. J. S., Kalinowski, A., Gregg, S. Q., Opresko, P. L., … Dahl, K. N. (2011). Computational image analysis of nuclear morphology associated with various nuclear-specific aging disorders. Nucleus, 2(6), 570–579. http://doi.org/10.4161/nucl.2.6.17798

Maize ear, cob and kernel shape attributes

Miller, N. D., Haase, N. J., Lee, J., Kaeppler, S. M., de Leon, N. and Spalding, E. P. (2017), A robust, high-throughput method for computing maize ear, cob, and kernel attributes automatically from images. Plant J, 89: 169–178. doi:10.1111/tpj.13320

Kernel size and shape analysis.(a) Example of raw image data with computed contour, kernel depth (major axis), and kernel width (minor axis) overlaid.(b) Features used in the statistical modelingapproach to finding the tip of each kernel, which produced the automated measurements of kernel area, length, and width.(c) Principal components analysis of the kernel contour data sets enabled their shapes to be described by three principal components PC1, PC2, and PC3. Changing PC1 causes the kernel to change size similarly in all directions. Changing PC2 changes eccentricity, or the ratio of the minor and major axes, while changing PC3 changes angularity of the contour.

Maize ear, cob and kernel shape attributes

Miller, N. D., Haase, N. J., Lee, J., Kaeppler, S. M., de Leon, N. and Spalding, E. P. (2017), A robust, high-throughput method for computing maize ear, cob, and kernel attributes automatically from images. Plant J, 89: 169–178. doi:10.1111/tpj.13320

Maize ear size and shape analysis.(a, b) Image processing steps convert the color image (a) to a value scale image (b).(c) After binarizing the image, a bounding box is fit to the ear object to obtain its length and maximum width.(d) Manually measured ear and cob lengths agreed almost perfectly with the values obtained automatically by the algorithm-based method.(e) Manually measured ear and cob widths agreed almost perfectly with the automated algorithm-based length measurements.(f) The ear contours determined from the binary mask (c) were subjected to principal components analysis to produce a set of three of basis vectors and the corresponding principal components PC1, PC2, and PC3. Sweeping PC1 primarily changed ear length. Sweeping PC2 primarily changed ear width, while sweeping PC3 controlled ear taper.

Identify, delineate and track stem cells

http://drexel.edu/now/archive/2015/September/CloneView/

Tracking moving / growing objects

http://www.albany.edu/celltracking/algorithms.html

https://youtu.be/WUIDQ1Ik7dw http://www.codesolorzano.com/software/CellFrost/

https://youtu.be/WUIDQ1Ik7dw

https://youtu.be/WUIDQ1Ik7dw

Root growth and traits - SmartRoot

SmartRoot: https://smartroot.github.io/

Multi-channel images

CellProfiler 2.1.0 manual: http://cellprofiler.org/getting_started/

ImageJ Sample images

Classification by supervised machine learning

http://jcs.biologists.org/content/126/24/5529

Pixel level

Cell objects

Whole images

Trainable segmentation – machine learning

http://forum.imagej.net/t/particle-segmentation/5110/4

No two workflows are the same…

http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0148469

…its the results that matter

http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0148469

Some examples of CIA being applied inFaculty of Science & Engineering

Examples of CIA applied in S&E projects

• Collagen fibre orientation and waviness analysis

– ImageJ / Fiji plugins and macro scripts:

• OrientationJ

• NeuronJ

• Quantification of mineral distribution in ore particles

– Avizo workflow and quantification

• Aquifer porosity analysis

– ImageJ plugins and macro scripts

Collagen fibre orientations

Wu, J.-P., Swift, B. J., Becker, T., Squelch, A., Wang, A., Zheng, Y.-C., Zhao, X., Xu, J., Xue, W., Zheng, M., Lloyd, D. and Kirk, T. B. (2017), High-resolution study of the 3D collagen fibrillary matrix of Achilles tendons without tissue labelling and dehydrating.Journal of Microscopy Volume 266, Issue 3, 2017. doi:10.1111/jmi.12537

Fibre orientations in cartilage

He, B., Wu, J. P., Chen, H. H., Kirk, T. B. and Xu, J. (2013), Elastin fibers display a versatile microfibril network in articular cartilage depending on the mechanical microenvironments. J. Orthop. Res., 31: 1345–1353. doi:10.1002/jor.22384

Fibre orientations in tendons


Fibre orientations in tendons


Collagen fibre waviness and global angle

Trace number Fibre length, Lf Straight line length, L0 Straightness parameter, Ps Global angle, a

N1 643.00 620.95 0.9657 29.20

N2 1029.44 987.53 0.9593 21.94

N3 1034.28 1000.69 0.9675 23.31

N4 1029.21 977.16 0.9494 26.43

N5 881.20 850.83 0.9655 14.43

N1N2N3 N4

N5

a(-) a(+)

Axial direction

Ps = L0 / Lf

Images courtesy of Mr Anas Almakhzoomi, PhD Candidate, Bioengineering Laboratory, Mechanical Engineering, Curtin University

Quantification of mineral distribution in ore particles

Rock pieces separated and labelledand size & volume fractions quantified

microCT image data

CT Image data courtesy of A/Prof Nimal Subasinghe, MEME, WASM, Curtin University. Labelled image courtesy of Yiyang Gao, Pawsey Internship final report.

Quantification of mineral distribution in ore particles

Histogram of mineralised ore particle

Histogram of waste rock particle

Mineral threshold

Total percentage of minerals vs mineral threshold

Perc

enta

ge o

f m

iner

als

in s

amp

le

Images courtesy of Yiyang Gao, Pawsey Internship final report, 2017.

Porosity estimation from microCT

Squelch, A., Harris, B. and AlMalki, M. Estimating porosity from CT scans of high permeability core plugs, ASEG Extended Abstracts 2012(1) 1 – 3, CSIRO Publishing. 2012.


ZoneWidth

(pixels)Height (pixels)

Area (pixels)

Porosity from image

analysis

Porosity from image histogram

1 670 123 82410 0.333 0.337

2 670 210 140700 0.389 0.388

3 670 158 105860 0.360 0.360

4 670 179 119930 0.394 0.393

All 670 670 448900 0.373 0.374

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 50 100

Po

rosi

ty

Grey-scale intensity

490 All

100 All

Zone 1

Zone 2

Zone 3

Zone 4

Squelch, A., Harris, B. and AlMalki, M. Estimating porosity from CT scans of high permeability core plugs, ASEG Extended Abstracts 2012(1) 1 – 3, CSIRO Publishing. 2012.


= -

Bringing images to life - 3D and lenticular prints

Pawsey compute resources

• Magnus• Athena

– Xeon Phi 7210 CPUs & Tesla P100 Pascal GPUs– Advanced Technology Cluster– Data analytics– Deep learning

• Nimbus– AMD CPUs– no GPUs (yet)– Cloud computing– Application clusters– Scientific data analytics and data workflow integration

THANK YOU!Questions / Discussion

Documents

-the Computational Image Analysis role