Fromfacedetec,ontothefacesofscien,ficimages:

ScalingAnaly,csforImageDatafromExperiments

LawrenceBerkeleyNa.onalLaboratory,Berkeley,CA,USA

DaniUshizima,HarinarayanKrishnan,TalitaPerciano,DulaParkinson,PeterErcius,WesBethelandJamesSethian

DataAnaly,cs&Visualiza,onDAVGroup

Wes Bethel, Daniela Ushizima, Gunther Weber, Dmitriy Morozov, Hank Childs, Talita Perciano, Mark Howison, Oliver Ruebel, Burlen Loring, David Camp, Hari Krishnan

2

Collaborators

CustomUI

DomainProcessing

Embedded

Lightweight,Collabora,on

TailoredVis

ClimateScience•  Customizeuserinterfaces:

–  Interface–Lat/Long2DGrid,3Dglobe,Con,nentalOverlays.–  Op,ons–ZonalMeanAverages,ExtremeValues,PeaksOverThreshold,etc..

•  Collabora,onandProvenance:–  Collaborate,Control,&CommunicateresultwithpeersRecordandrecreateworkflows.

•  ExtendCapabili,es:–  ExtendExtremeValueAnalysisorPeaks-Over-Thresholdalgorithmorwritecustomanalysisrou,nesto

exploredata.

DataBrowser(SDM,ACS),DeepVadoseZone(PNNL),JohnPeterson,SusanHubbardEnvironmentalScience

Astrophysics(YT),Climatescience(R),VTK(python)

Astrophysics

6/16/16 8

ImageProcessing,Reconstruc,on,Segmenta,on,andAnalysis

Restofthetalk…

Overview1.  Inves,ga,ngimage-basedexperiments:

a.  MaterialScience-focusedimageanalysis;

b.  Health-focusedimageanalysis;

2.  Computermethodsandresults;

3.  Scalingthroughpartnerships;4.  Imageintheexascalelandscape

6/16/169

OURTOOLS

CAMERACenterforAppliedMathema,csforEnergyResearchApplica,ons

FractureAnalysisofHigh-resImages 10

6/16/16 11

6/16/1612

Nanoparticle Ocular fundus Head CT Radar image

6/16/16 13

SAIDE projects – from nano to meter scale

SIAM2010

ISBI’14+15

AdvancedFunc.Materials2015

UXMagazine2013 PSOC-NCI2011

Real->meImagingActaMicroscopica ACS2014

DemoNCEM2013 DemoESD2012

DemoLSD2013

DemoEETD

IEEEBigData2014

Chemical+

Electronic+

Structural

Chemical+

Structural StructuralChemical

+Structural

Chemical+

Structural

Chemical+

Structural

Chemical+

Structural

Chemical+

StructuralStructural Structural

Chemical+

Structural

Chemical+

Structural

ImageAcrossDomains

14

Specimens• Materials,composites,compoundsandbiologicalsamples.

Formats• Tiff,jpeg,hdf5,featurevectors,mul,-resolu,onpyramids,binaries.

DataAnalysis• Morphometry;• Spectralcontent;

• Mul,modal;• Templates.

DataUnderstanding• Clustering;• Classifica,on;• Randomizedschemes;

• Visualiza,on.

Reproducibleresearch• Datarepositories;• Sokwarerepositories;

• Collabora,on.

1.Imageanalysis@UCB-BIDS/LBL

6/16/16

Geologicalsamples

Resistantcomposites

Free-sokware,open-source,git,reproducible

Cervicalcells

Pilliden,fier

1.a.Imageanalysis@LBL/UCB-BIDS

6/16/16

Geologicalsamples

Resistantcomposites

ECRP,CAMERAandDAV

•  Scidac2012– Geologicalsamples– Carbonsequestra,on

•  MathFoundry2013– MicroCT-imagedsamples– ConfocalandPS-OC

•  CAMERA2014– ASCR+BES

6/16/16 17

rock

bone

composite

The science question: material resilience sample (CMC) and instrument (microCT) ▪  DetectcracksandfiberbreaksfrommicroCTimagesfromALSto

quan.fytherobustnessandresilienceofnewmaterials:noautomatedmethodsexistforthistypeofanalysis;

▪  Constraints:(1)exis,ngsokwaretoolsincapableofmee,ngthroughputrequirementsandscaletofull-resolu,onoftheexperiment(raw~60GB)(2)unabletoprovidereal-,mefeedback.

WorkwasperformedatLawrenceBerkeleyNa,onalLaboratorybytheCRDCenterforAppliedMathema,csinEnergyResearchApplica,ons(CAMERA)andonALSBeamline8.3.2.Opera,onoftheALSissupportedbyU.S.DepartmentofEnergy,OfficeofBasicEnergySciences.CAMERAissupportedbyjointlybyU.S.DepartmentofEnergy,

AdvancedScien,ficCompu,ngResearchandOfficeofBasicEnergySciences.

t

Pressure & temperature

Micro-CT Pattern RecognitionProblem: quantify micro-structural damage of ceramic matrix composites using time-resolved data for full exploration of the micro-tomographycontent; Goal:-  Iden,fymaterialfailureanddeformi,esfrommicro-CT,forexample,to

inspectfiberreinforcedCMC,anddendritespermea,ngbaseries;-  Real-,mefeedbackaboutdatacollec,onandsamplecondi,on;

Approach: •  Develop scalable pattern

recognition algorithms to find defects from 3D images;

•  Createsokwaretoolstobeserinterfacehumanstoinstrumentswithhighresolu,onhigh-throughput.

19

19

DOEEarlyCareerResearchProject:ScalingAnaly.csforImageDatafromExperiments(SAIDE)D.Ushizima(P.I.),T.Perciano,H.Krishnan,D.Parkinson(ALS),R.Richie(UCB),E.W.Bethel(LBNL)&J.Sethian(CAMERA)

FractureAnalysisofHigh-resImages 20

93N 133N 151N

Deformation evolution

FractureAnalysisofHigh-resImages

approachTemplate matching

Apply F3D filtersto improve contrast

to extract compositeApply F3D filters

Template matching

Intersection with"Base Result"

with high tolerance

Template matching with low tolerance

Union

For each slice in the stack Prototype examples

Identification of structures

Rawdata

Templatematching

FractureAnalysisofHigh-resImages 22

MSE(x, y) = 1n

[p(i, j)− f (x + i, y+ j)]2i, j∑

NCCC(x, y) =p(i, j)− p(i, j)

i, j∑ f (i, j)− f (i, j)

i, j∑

( p(i, j)− p(i, j)i, j∑ f (i, j)− f (i, j)

i, j∑ )2

#

$%%

&

'((

12

1)Similaritybetweenprototypesandlocalregions:

2)Determinethebestmatches:

approachTemplate matching

Apply F3D filtersto improve contrast

to extract compositeApply F3D filters

Template matching

Intersection with"Base Result"

with high tolerance

Template matching with low tolerance

Union

For each slice in the stack Prototype examples

F3Dplugin•  Accelerate key image

processingalgorithms•  Enablesegmenta,onand

analysisofhighresolu.onimagedatasets

•  Requirement:parallel-capablealgorithmstoaccommodatelargedatasizesandtoallowreal-,mefeedback

hVps://github.com/CameraIA/F3D

•  Non-linearedgepreservingfilters•  Morphologicaloperatorswithvaryingstrel

Image processing at high-resolution

DOEEarlyCareerResearchProgram

FractureAnalysisofHigh-resImages 24

Quantitative results

FractureAnalysisofHigh-resImages 25

17Xfaster

●●●●●●●●●●● ●● ● ● ● ●

●●

●

020

4060

80

Performance

Data Size (Gb)

Tim

e (m

in)

0 2 5 7 12 19 30

●●●●●●●●●●● ●● ● ● ● ●●

●

●

●●●●●●●●●●● ●●●

●

●

●

●

●

●

F3DF3D Virtual StackSacha

Performance evaluation: comparison between proposed filter and only tool previously available in Fiji

•  IntelXeonCPUE5-2660-20GHz•  3NVIDIATeslaK20X+1K40m

Terabyte-sizeimagerepresenta,on•  Problem:

–  Largedatasets(originally16GBperframe)•  Solu,on:

–  Mul,resolu,onpyramidsatfourdifferentscalesstoredasHDF5chunkedmul,-dimensionalarraysthroughBig-DataViewer;

–  Pluginoriginallyoffersinterac,vearbitraryvirtualreslicingofmul,-terabyterecordings,sothattheusercaninspecttheexperimentaldataefficiently;

–  Compressfilesandallowencapsula,onofterabyte-sizeimagedatasets,includingmetadata,andop,mizedaccesstomul,plescalesofthedata,bothforvisualiza,onaswellasforprocessing.

–  OtheradvantagesofBigDataViewerformaung:a)increasedcompu,ngperformance,b)decreasedcluseringoftheexperimentalarchives,andc)poten,alforparallelI/O.

6/16/16 26

Ref:T.Pietzsch,S.Saalfeld,S.Preibisch,andP.Tomancak.Bigdataviewer:visualiza,onandprocessingforlargeimagedatasets.NatureMethods,2015.

Tes,ngfileswithdifferentsizes

6/16/16 27

Scalabilityofthemul,-dimensionalrepresenta,onusingHDF5withincreasingdatasize.

Advanced technique: team work

DOEEarlyCareerResearchProgram

6/16/16 29

Inven,ngnewcodesforcharacteriza,onofreinforcedcomposites

SignificanceandimpactScien=ficAchievement§  Analysisofthinfilmsbyusingscanning

transmissionelectronmicroscopy(STEM)tomographyimagesinsupportofmaterialarchitectureenhancement;

§  Quan,fyporestructureevolu,oninordertocontrolqualityoffabricatedfilms.

§  ResultsusingporosimetryfromSTEMimagescorroboratediniden,fica,onoffabrica,oncondi,onsthatledtothelowesteverdielectricconstantsfortheneededfilms.

§  Collabora,onwithIntel,LBLNCEMandOrganicandMacromolecularSynthesisattheMolecularFoundry,andSLAC,

Researchdetails*§  ReportedlowesteverdielectricconstantsforPMOmatrix

material,usedinmicroelectronics;§  Textureanalysisusingsecond-ordersta,s,csofimageintensity

varia,onstomeasurefilmroughness;§  Newtoolsadaptedto3DstacksforNCEMinstruments;§  Newdevelopments:porosityanalysisusingnewmaterial

architecturedrivers(withT.WilliamsandB.Helms)andspectralanalysisofcataly,cprocesses(withK.Bus,lloandP.Ercius).

Ref:Willsetal,“BlockCopolymerPackingLimitsandInterfacialReconfigurabilityintheAssemblyofPeriodicMesoporousOrganosilicas”,FuncionalMaterials2015.

Imageanalysisforqualitycontrolofmaterialarchitecture

Image-based porosimetry for quality control during assembly of films CAMERA and Molecular Foundry

*WorkwasperformedatLBNLbytheCRDDAVandCAMERA.DAVissupportedbyASCRandCAMERAjointlybyASCRandBES.

FeaturedesignforSTEMimagedata

(A)(bluetraces)and(B)(redtraces).

DOEEarlyCareerResearchProgram

Finalremarks

•  Scalingthroughpartnerships:

6/16/16 32

Finalremarks

•  Algorithmsinanexascalelandscape–  I/Oawareness

– Datareduc,onandin-situanalysis

– Machinelearning

– Experimental/observa,onaldatasets

– Digitaltwin

6/16/16 33