NEW ALGORITHMS FOR COMPRESSED SENSING OF MRI: …

Kennesaw State UniversityDigitalCommons@Kennesaw State University

Master of Science in Computer Science Theses Department of Computer Science

Spring 6-6-2018

NEW ALGORITHMS FOR COMPRESSEDSENSING OF MRI: WTWTS, DWTS,WDWTSSrivarna Settisara Janney

Follow this and additional works at: https://digitalcommons.kennesaw.edu/cs_etd

Part of the Bioimaging and Biomedical Optics Commons, Biomedical Commons, and the SignalProcessing Commons

This Thesis is brought to you for free and open access by the Department of Computer Science at DigitalCommons@Kennesaw State University. It hasbeen accepted for inclusion in Master of Science in Computer Science Theses by an authorized administrator of DigitalCommons@Kennesaw StateUniversity. For more information, please contact [email protected].

Recommended CitationSettisara Janney, Srivarna, "NEW ALGORITHMS FOR COMPRESSED SENSING OF MRI: WTWTS, DWTS, WDWTS" (2018).Master of Science in Computer Science Theses. 11.https://digitalcommons.kennesaw.edu/cs_etd/11

https://digitalcommons.kennesaw.edu?utm_source=digitalcommons.kennesaw.edu%2Fcs_etd%2F11&utm_medium=PDF&utm_campaign=PDFCoverPages

https://digitalcommons.kennesaw.edu/cs_etd?utm_source=digitalcommons.kennesaw.edu%2Fcs_etd%2F11&utm_medium=PDF&utm_campaign=PDFCoverPages

https://digitalcommons.kennesaw.edu/cs?utm_source=digitalcommons.kennesaw.edu%2Fcs_etd%2F11&utm_medium=PDF&utm_campaign=PDFCoverPages

https://digitalcommons.kennesaw.edu/cs_etd?utm_source=digitalcommons.kennesaw.edu%2Fcs_etd%2F11&utm_medium=PDF&utm_campaign=PDFCoverPages

http://network.bepress.com/hgg/discipline/232?utm_source=digitalcommons.kennesaw.edu%2Fcs_etd%2F11&utm_medium=PDF&utm_campaign=PDFCoverPages




https://digitalcommons.kennesaw.edu/cs_etd/11?utm_source=digitalcommons.kennesaw.edu%2Fcs_etd%2F11&utm_medium=PDF&utm_campaign=PDFCoverPages

mailto:[email protected]

1

NEW ALGORITHMS FOR COMPRESSED SENSING OF MRI:

WTWTS, DWTS, WDWTS

An Abstract of

A Thesis Presented to

The Faculty of Department of Computer Science

by

Srivarna Settisara Janney

In Partial Fulfillment

of Requirements for the Degree

Master of Science

Kennesaw State University

May 2018

2

ABSTRACT

Magnetic resonance imaging (MRI) is one of the most accurate imaging

techniques that can be used to detect several diseases, where other imaging

methodologies fail. MRI data takes a longer time to capture. This is a pain taking

process for the patients to remain still while the data is being captured. This is

also hard for the doctor as well because if the images are not captured correctly

then it will lead to wrong diagnoses of illness that might put the patients lives in

danger. Since long scanning time is one of most serious drawback of the MRI

modality, reducing acquisition time for MRI acquisition is a crucial challenge for

many imaging techniques. Compressed Sensing (CS) theory is an appealing

framework to address this issue since it provides theoretical guarantees on the

reconstruction of sparse signals while projection on a low dimensional linear

subspace. Further enhancements have extended the CS framework by performing

Variable Density Sampling (VDS) or using wavelet domain as sparsity basis

generator. Recent work in this approach considers parent-child relations in the

wavelet levels.

This paper further extends the prior approach by utilizing the entire

wavelet tree structure as an argument for coefficient correlation and also considers

the directionality of wavelet coefficients using Hybrid Directional Wavelets

(HDW). Incorporating coefficient thresholding in both wavelet tree structure as

well as directional wavelet tree structure, the experiments reveal higher Signal to

Noise ratio (SNR), Peak Signal to Noise ratio (PSNR) and lower Mean Square

Error (MSE) for the CS based image reconstruction approach. Exploiting the

sparsity of wavelet tree using the above-mentioned techniques achieves further

lessening for data needed for the reconstruction, while improving the

reconstruction result. These techniques are applied on a variety of images

including both MRI and non-MRI data. The results show the efficacy of our

techniques.

3

NEW ALGORITHMS FOR COMPRESSED SENSING OF MRI: WTWTS, DWTS, WDWTS

A Thesis Presented to

The Faculty of Department of Computer Science

by

Srivarna Settisara Janney

In Partial Fulfillment

of Requirements for the Degree

Master of Science

Advisors: Dr. Sumit Chakravarty and Dr. Chih-Cheng Hung


May 2018

4

NEW ALGORITHMS FOR COMPRESSED SENSING OF MRI: WTWTS, DWTS, WDWTS

Approval Committee Members: Dr. Chih-Cheng Hung, Thesis Director and Advisor Department of Computer Science Kennesaw State University Dr. Sumit Chakravarty, Advisor Department of Electrical Engineering Kennesaw State University Dr. Craig Chin Department of Electrical Engineering Kennesaw State University Dr. Mingon Kang Department of Computer Science


5

DEDICATION

This thesis is dedicated to my husband who has been my inspiration and my strongest supporter. I would also thank my parents, in-laws, brother & his family, and sister-in-law & her family.

6

PREFACE

The thesis is submitted in partial fulfillment of the requirements for the Master of Science degree program at Kennesaw State University, Kennesaw.

The research project has been conducted under the supervision of Dr. Sumit Chakravarty from Department of Electrical Engineering and Dr. Chih-Cheng Hung from Department of Computer Science, during the years 2017-2018. Financing for the work has been provided in the form of scholarship from the Graduate College of Kennesaw State University through the Graduate Research Assistantship.

7

ACKNOWLEDGEMENTS

I would like to express my gratitude to my advisors Dr. Sumit Chakravarty and Dr. Chih-Cheng Hung for their support, encouragement, and motivation through this entire thesis. Thank you for believing in me and leading me to the path of research. I would also like to thank my thesis committee members, Dr. Mingon Kang and Dr. Craig Chin for their insightful comments and valuable suggestions.

8

QUOTES

“Arise,Awake,andstopnotuntiltheGoalisachieved”

- SwamiVivekananda

“Theimportantthingistonotstopquestioning.Curiosityhasitsownreasonfor

existing.”-AlbertEinstein

“Weallneedpeoplewhowillgiveusfeedback.That’showweimprove.”-BillGates

9

TableofContents

Chapter1............................................................................................................141.1 Introduction............................................................................................................................................141.1.1 MagneticResonanceImaging(MRI).......................................................................................151.1.2 BasicConceptsofCompressedSensing...................................................................................16

Chapter2............................................................................................................212.1 Literaturesurveyandproblemstatement................................................................................212.1.1 CompressedSensingforMRI.......................................................................................................212.1.2 TraditionalCompressedSensingMethodologiesinMRI.................................................242.1.3 Drawbacks.........................................................................................................................................282.1.4 ProblemStatement.......................................................................................................................28

Chapter3............................................................................................................293.1 ProposedSolutions..............................................................................................................................293.1.1 WeightedThresholdWaveletTreeStructure(WTWTS)................................................303.1.2 DirectionalWaveletTreeStructure(DWTS).......................................................................323.1.3 WeightedDirectionalWaveletTreeStructure(WDWTS).............................................35

3.2 ComplexityAnalysis............................................................................................................................37

Chapter4............................................................................................................384.1 Experimentsandresults...................................................................................................................384.1.1 TestEnvironmentSpecification.................................................................................................384.1.2 DataSet................................................................................................................................................384.1.3 Hyper-Parameters...........................................................................................................................414.1.4 Parameters..........................................................................................................................................424.1.5 PreliminaryResults.........................................................................................................................424.1.6 ResultsandEvaluation..................................................................................................................524.1.7 ComparisonbetweenWTWTS,DWTS,WDWTS.................................................................58

Chapter5............................................................................................................59FutureWork.......................................................................................................................................................59

Chapter6............................................................................................................63Conclusion...........................................................................................................................................................63Bibliography.......................................................................................................................................................64Appendix..............................................................................................................................................................72ListofAlgorithms.............................................................................................................................................72

10

ListofFigures

Figure1.1:MRIscanning..............................................................................................................................14Figure1.2:MRIscanningmachine[2]....................................................................................................16Figure1.3:CompressedSensingvsImageCompressionFramework[3]...............................17Figure1.4:BasicCompressedSensingmatrix[3].............................................................................18Figure1.5:Comparisonbetween(a)l1issparseand(b)l2normsisnotsparse...............19Figure1.6:(a)kspacedata(b)kspacedatawithnoiselikepatterncausebyincoherence

........................................................................................................................................................................19Figure2.1:ImageReconstructionfromk-spacetopixelimage[40]........................................22Figure2.2:RepresentstheoriginalimageanditsreconstructedimageusingSVD

method.......................................................................................................................................................25Figure2.3:(a)OriginalImage(b)CSwithPCA,redcirclemeanstheareahashigherSNR

comparedtopreviousmethods.......................................................................................................26Figure2.4:(a)WaveletDecomposition(b)DecomposedImagedirection............................27Figure3.1:WaveletQuad-TreeStructure(a)AcardiacMRimage(b)Corresponding

WaveletTreeGroupofHierarchyGP-P-CStructure...............................................................30Figure3.2:Directionalfilterbankfrequencypartitioningusingeightdirections...............32Figure3.3:(a)Verticaldirectionalfilterbanks(b)Horizontaldirectionalfilterbanks...32Figure3.4:ExampleofDirectionalfilterbankonrepresentation..............................................33Figure3.5:EightDirectionalfilterbanksrepresentusingclassicBarbaraImage..............33Figure4.1:MRIData.......................................................................................................................................39Figure4.2:SamplingMask...........................................................................................................................39Figure4.3:(a)T1imageand(b)T2image...........................................................................................40Figure4.4:PortraitData...............................................................................................................................40Figure4.5:CrowdData..................................................................................................................................41Figure4.6:LandscapeData.........................................................................................................................41Figure4.7:VariationofnumberofzerotreesbasedonGP-P-CHierarchyforonebrain

image...........................................................................................................................................................43Figure4.8:VariationofnumberofzerotreebasedonGP-P-CHierarchyforT1images.43Figure4.9:VariationofnumberofzerotreesbasedonGP-P-ChierarchyforT2image.44Figure4.10:VariationofnumberofzerotreesbasedonGP-P-Chierarchylevelforentire

500brainimages....................................................................................................................................44Figure4.11:3DGraphofrangeofthresholdsetsagainstEntropyofdataatParent-Child

Level.............................................................................................................................................................45Figure4.12:ComparisonofdifferentthresholdingsetsagainstSNRfordifferent

resolutionsizesforBrainImages(RedlineindicatereferenceSNRofWatMRIalgorithm)..................................................................................................................................................46

Figure4.13:ComparisonofdifferentthresholdingsetsagainstSNRfordifferentresolutionsizesforHeartImages(RedlineindicatereferenceSNRofWatMRIalgorithm)..................................................................................................................................................46

Figure4.14:ComparisonofdifferentthresholdingsetsagainstSNRfordifferentresolutionsizesofShoulderImages(RedlineindicatereferenceSNRofWatMRIalgorithm)..................................................................................................................................................46

11

Figure4.15:ComparisonofdifferentthresholdingsetsagainstSNRfordifferentresolutionsizesforChestImages(RedlineindicatereferenceSNRofWatMRIalgorithm)..................................................................................................................................................47

Figure4.16:SNRcomparisonbetweenWatMRI(Greencolor)andWTWTSwithSet2(Redcolor)and3(Bluecolor)approaches.................................................................................47

Figure4.17:ComparisontheSNRforPortrait,CrowdandLandscapeimagesforthresholdingset2and3(RedlineindicatesreferenceSNRofWatMRIalgorithm)48

Figure4.18:Histogramof8DirectionsatParentandChildlevelforBrainImage.............48Figure4.19:Histogramof8DirectionsatParentandChildlevelforHeartImage.............49Figure4.20:Histogramof8DirectionsatParentandChildlevelforShoulderImage.....49Figure4.21:Histogramof8DirectionsatParentandChildlevelforChestImage.............50Figure4.22:ComparisonofPortraitimageagainstoriginaldatausingWTWTSand

WDWTSalgorithmswithchosenthresholdset........................................................................51Figure4.23:ComparisonofCrowdimageagainstoriginaldatausingWTWTSand

WDWTSalgorithmswithchosenthresholdset........................................................................51Figure4.24:ComparisonofLandscapeimageagainstoriginaldatausingWTWTSand

WDWTSalgorithmswithchosenthresholdset........................................................................52Figure4.25:Brainimagereconstructionusingthe3algorithmscomparedagainst

WatMRI.......................................................................................................................................................53Figure4.26:Heartimagereconstructionusingthe3algorithmscomparedagainst

WatMRI.......................................................................................................................................................53Figure4.27:Shoulderimagereconstructionusingthe3algorithmscomparedagainst

WatMRI.......................................................................................................................................................54Figure4.28:Chestimagereconstructionusingthe3algorithmscomparedagainst

WatMRI.......................................................................................................................................................54Figure4.29:Portraitimagereconstructionusingthe3algorithmscomparedagainst

WatMRI.......................................................................................................................................................55Figure4.30:Crowdimagereconstructionusingthe3algorithmscomparedagainst

WatMRI.......................................................................................................................................................55Figure4.31:Landscapeimagereconstructionusingthe3algorithmscomparedagainst

WatMRI.......................................................................................................................................................56Figure5.1:OverviewofDeepNeuralNetworkforCSinMRI.......................................................59Figure5.2:ReconstructionErrorflowchart........................................................................................60Figure5.3:Multi-SliceofBrainMRI[38]...............................................................................................60Figure5.4:3DBrainMRIdisplayingeachdimension[40]............................................................61Figure5.5:DynamicMRIrepresentedinTensor[46].....................................................................61Figure5.6:VisualizationofHigherOrderSingleValueDecomposition(HOSVD)(a)3D

Tensor(b)4DTensor[45].................................................................................................................62

12

ListofTables

Table3.1:PseudoCodeofWTWTS..........................................................................................................31Table3.2:PseudoCodeofDWTS..............................................................................................................34Table3.3:PseudoCodeofWDWTS..........................................................................................................36Table4.1:SNRandMSEcomparisononMRIdata............................................................................56Table4.2:CPUTimecomparisononMRIdata....................................................................................57Table4.3:SNRandMSEcomparisononNon-MRIdata..................................................................57Table4.4:CPUTimecomparisononNon-MRIdata..........................................................................58

13

ListofAcronyms

BMP:BasisMatchingPursuit.....................................................................................................................................22C:Child.....................................................................................................................................................................................29CS:CompressedSensing...............................................................................................................................................14DFB:DirectionalFilterBanks.....................................................................................................................................32DWTS:DirectionalWaveletTreeStructure....................................................................................................30,32FCSA:FastCompositeSplittingAlgorithm..........................................................................................................23FISTA:FastIterativeShrinkageThresholdingAlgorithm...........................................................................27GP:Grandparent.................................................................................................................................................................29HDW: Hybrid Directional Wavelets............................................................................................................................2MRI:MagneticResonanceImaging.........................................................................................................................14MSE: Mean Square Error..................................................................................................................................................2OMP:OrthogonalMatchingPursuit........................................................................................................................22P:Parent..................................................................................................................................................................................29PSNR: Peak Signal to Noise Ratio...............................................................................................................................2SNR: Signal to Noise Ratio.............................................................................................................................................2TVCMRI:TotalVariationCompressedSensingMRI......................................................................................23VDS: Variable Density Sampling.................................................................................................................................2WaTMRI:WaveletTreeMRI.......................................................................................................................................27WDWTS:WeightedDirectionalWaveletTreeStructure..........................................................................30,35WTWTS:WeightedThresholdWaveletTreeStructure....................................................................................30

14

Chapter1

1.1 Introduction

OneofthecorechallengesofMagneticResonanceImaging(MRI)

reconstructionisthesignificanttimedurationrequiredtoacquirethe

images.Oneoftheconstraintsisthatpatientshavetoremainstationaryfor

thedurationofscanthattypicallytakesmorethananhour.Another

constraintisthattheresolutionofthereconstructedimagesshouldbeof

significantlyhighgrade.InthecaseofMRI,considerableattentionhasbeen

paidtoCompressedSensing(CS)thatisknownasoneofthemostsuccessful

methodsforimagereconstructionwithsignificantlysparseamountof

coefficients.Figure1.1showstheMRIscanningprocessthatpatientshaveto

undergowhilegettingaMRIscan.

Figure1.1:MRIscanning

Initiallythepatientsaregivenaninjectioncontaining“Gadolinium”whichis

aheavymetaloralsocalledas“contrastagent”.Theywaitfor10-15minsfor

theinjectedcontenttospreadthroughouttheareathatneedtobescanned.

Patientisthenaskedtoliedownonaslidingtablethatentersthehuge

magneticcoildrumthatcapturesthescandatainspaceknownask-space

data.CompressedSensingofwaveletisusedtoaccuratelyreconstructthe

imagethatknownasMRIscanningimage.Thewholeprocesstypicallytakes

aroundtwohoursofpatient’stime.Ifthereisanymovementbythepatient,

15

theninaccuratereconstructiontakesplaceandprocessneedstoberepeated

forrecapture.Thisisachallengingsituationforbothpatientsandradiologist.

ThisresearchpaperdevelopedthreenewalgorithmsnamelyWeighted

ThresholdWaveletTreeStructure(WTWTS),DirectionalWaveletTreeStructure

(DWTS)andWeightedDirectionalWaveletTreeStructure(WDWTS),using

multipleMRIdataincludingBrain,Chest,Shoulder,andHearttypeofdatasets.

DevelopedalgorithmswerealsotestedonNon-MRIdata(Portrait,Landscape,

CrowdImages)togeneralizethealgorithmtoanyimagedata.

1.1.1 MagneticResonanceImaging(MRI)

MagneticResonanceImaging(MRI)scanisatechniqueusingby

radiologiststhatusesmagnetismofthehugemagneticcoildrumandalgorithms

areusedtoreconstructimageofbodystructures.Fortheprocedures,acontrast

agent,suchasgadolinium,whichisaheavymetal,isinjectedintothepatient’s

bodypartthatneedstoscan.TheMRIscannerisagiantcirculardrumofmagnet

andaslidingtableasshowninFigure1.2.Thepatientisplacedonamoveable

bedthatisinsertedintothemagnet.Themagnetcreatesastrongmagneticfield

thatalignstheprotonsofhydrogenatoms.Thisspinsthevariousprotonsofthe

body,andtheyproduceasignalthatisdetectedduringtheMRIscanner.A

computerprocessesthereceiverinformation,andaftercompressedsensing

usingwavelettoreconstructthefinalimage,knownasMRIisproduced.

TheimageandresolutionproducedbyMRIneedstobedetailedand

shoulddetecttinychangesofstructureswithinthebody.Theradiologistwill

ablebeaccuratelydiagnosediseasesorstructuralabnormalitiesbasedonthe

qualityofimagethatisreconstructedfromthemachine.Ifthepatientmoves

thenthedatawillnotbeaccuratelycapturedandtheradiologisthastoredothe

scanning.

16

Patientswhohaveanykindofmaterialsthatcontainmetalintheirbodies

willcreateaproblemduringscanning.Normallytheyareaskedtoremovethings

likejewelry,clipsetc.Mostoftheinternalmetalmaterialsthatareinsidethe

patient’sbodyaddedasapartofmedicalproceduressuchasartificialjoints,

pacemaker,metallicboneplates,orprostheticlimbsetc.coulddistortMRI.

Scanningcannotbeperformedonthesepatients.Alternativemethodswillbe

usedfordiagnosis[1].

Figure1.2:MRIscanningmachine[2]

Sincesomepatientscanbeclaustrophobicwhoexperiencefearofclosedplaces,

theradiologistmayrecommendamildsedativepriortotheMRIscanning.The

rotatingofmagneticcoilgenerateshugenoise,soearplugsmightbegivento

patientswhorequestthem.

1.1.2 BasicConceptsofCompressedSensing

17

CompressedSensingshouldnotbeconfusedwithtraditionalimage

compression.Figure1.3showstheframeworkofthetwoconceptswhereXisthe

originalinputsignalandXcisthereconstructedsignalwithsampleNnumberof

sampleswhileMrepresentsthesparsenumberofsamplethatisrequiredfor

reconstruction.

Figure1.3:CompressedSensingvsImageCompressionFramework[3]

Compressedsensing(alsoknownascompressivesensing,sparsesamplingor

compressivesampling,CS)isasignalprocessingtechniqueforefficiently

collectingsignalandaccuratelyreconstructingthesignal.Basedontheprinciple

ofsparsityofthesignalfound,thesesignalshavemuchfewersamplesthanthat

requiredbytheShannon-Nyquistsamplingtheorem[4],[5],[6],[7].

CShasgainedalotofpopularityvariousareaoverlastfewdecades.This

conceptwasintroducedbyCandes,Romberg,andTao[12],andDonoho[14].CS

helpstorecoverthesparsesignalsusingonlyfewsamplesincontrasttoNyquist

samplingtheory,wheresamplingofthesignalisperformedataratelargerthan

highestfrequencypresentinthesignal.Anoverviewofcompressedsensing

theorycanbefoundin[8],[9],[10],[11].

Consider𝑏tobetheactualsignalhavingdimensionsNx1where𝑏 ∈ ℝ.Φ

18

representsthesamplingmatrixwithMxNdimensionsandyisthefinal

compressedvectorwithdimensionMx1whichismuchsmallerthanN.i.eM<<N

𝑦 = Φ𝑏(1)

AccordingtoCompressedSensing,theequationlookslike

𝑦 = ΦΨ𝓍=A𝓍(2)

wherebisthenon-sparsesignalrepresentedas𝑏=Ψ𝓍,𝓍=sparse

representationintransformdomainΨ.A=ΦΨwhichrepresentsdimension

MxNsensingmatrixhavingksparsemeasurementrequiredforreconstructionof

image.Theaimistominimizecoefficientstogetsparsesignalcalledmin! .Figure

1.4showsthefundamentalconceptsinCSmatrixrepresentationthatisshownin

equation(2).

Figure1.4:BasicCompressedSensingmatrix[3]

TherearetwofundamentalconceptsinCompressedSensing.Firstone

issparsityofthesignal;thatis,thesignalneedstosparseintransformdomain.

Thesecondoneisincoherence;thatis,appliedthroughtheRestrictedIsometric

Property(RIP)thatissufficientforsparsesignals.

Theabovereconstructionproblemisnon-deterministicpolynomial-time

19

hard(NP-hard).Duetothenon-convexnatureof𝑙!minimization,theproblemis

illconditionedanddifficulttosolveasitrequiresanexhaustivesearchtofindthe

mostsparsesolution(i.e.combinatorialproblem)[16].Sincetheuniquesparse

solutioncanbefoundexactlyusing𝑙!minimization.𝑙!normisknownforleast

squaresolution,apenaltywillbeimposedonsmallnon-zerocoefficientswhere

ittendstoenergytolargenumberofentriesandtheresultisnotsparse[13].

ComparisonbetweenthenormsisasshowninFigure1.5andtheincoherenceis

showninFigure1.6.

(a)(b)

Figure1.5:Comparisonbetween(a)l1issparseand(b)l2normsisnotsparse

(a)(b)Figure1.6:(a)kspacedata(b)kspacedatawithnoiselikepatterncausebyincoherence

20

Inadditiontosparsity,theminimumacceptablenumberofsamplesfor

reconstructionalsodependsupontheincoherencebetweenΦ andΨ,suchthat

thereisnocorrelationbetweentherowsofΦandthecolumnsofΨ.Toachieve

this,“A”asexplainedintheequation(2)shouldsatisfytheRestrictedIsometry

Property(RIP)[17].

1.1.2.1 Disadvantages

Earlyiterationsmayfindinaccuratesampleestimates,howeverthis

methodwilldown-sampletheseatalaterstagetogivemoreweighttothe

smallernon-zerosignalestimates.Oneofthedisadvantagesistheneedfor

definingavalidstartingpoint,asaglobalminimummightnotbeobtainedevery

timeduetotheconcavityofthefunction.Anotherdisadvantageisthatthis

methodtendstouniformlypenalizetheimagegradientirrespectiveofthe

underlyingimagestructures.Thiscausesover-smoothingofedges,especially

thoseoflowcontrastregions,subsequentlyleadingtolossoflowcontrast

information[3].

1.1.2.2 Advantages

Theadvantagesofthismethodinclude:reductionofthesamplingratefor

sparsesignals;reconstructionoftheimagewhilebeingrobusttotheremovalof

noiseandotherartifacts;anduseofveryfewiterations.Thiscanalsohelpin

recoveringimageswithsparsegradients[3].

21

Chapter2

2.1 Literaturesurveyandproblemstatement

Thissectionisrelatedtopreviousresearchesthathavebeencarried

out.Eachsub-sectionshowcasestheidea,equationandthescopeof

improvementthatbeenbroughtaboutthroughthefindings.Theseconcepts

markabaselineforthisresearchisbuiltoversomeoftheseideas.

2.1.1 CompressedSensingforMRI

CompressedSensing(CS)isoneofthemostsuccessfulmethodsof

imagereconstructionwithsignificantlysparseamountofcoefficients.

AccordingtoCStheory[15][19],onlyO(K+Klogn)sampling

measurementsareenoughtorecoverK-sparsedatawithlengthn,the

probleminovercomingthechallengeassociatedwithimagingwithouta

significantdegradationinimagequality.Thekeyapproachofcompressed

sensingMRIapplicationsistoutilizeseparatedomains.AsMRIimagesare

formedinK-space(FourierDomain)itbecomestheprimarydomainfor

monitoringreconstructionerrorasshownintheFigure2.1.Ithasbeenwell

establishviatheresearchofLustigandDonoho[19]toimplementsparsity

thatneedstooperateinasparsifyingdomainlikewavelets.Finallyastheend

consumersoftheimagedata,theneedistovisualizetheresultinthespatial

sampledomain.

22

Figure2.1:ImageReconstructionfromk-spacetopixelimage[40]

Thenon-linearconjugategradient(NLCG[18])wasappliedtosolve

theunconstrainedproblemintheLagrangianformtogetequation(3):

𝑚𝑖𝑛!{ 𝐹 𝑥 = !!𝐴𝑥− 𝑏 !

! + 𝛽 𝜙! !} (3)

Where𝐴=samplingmatrixobtainedfrompartialFouriertransformof

𝑥=MxNimagetobereconstructed,𝛽=positiveparameter,b=original

signaldataand𝜙=sparsifyingdomain,

Equation(3)attemptstominimizethemeansquareerrorinthe

frequencydomainwhilechoosingthesparsestsetofcoefficientsinthe

waveletdomainwithL2norm.Thisoptimizationcanbeperformedvia

varioustechniqueslikeOMP(OrthogonalMatchingPursuit)[21]orBasis

MatchingPursuit(BMP)[22]algorithms.

SeveralstudieshavebeendoneinCS-MRIdomain.Muchoftheworkdeals

23

withmanipulatingtheconstraintterminequation(3).Techniquesinthe

gamutofsignalprocessingliketotalvariationsmoothening,opticalflowof

coefficients,[23]variabledensitysampling[24]andsingularvalue

decomposition[25],conjugategradient[26]andpartialFourier(RecPF)[57]

aresomeoftheproposedapproachesusedincurrentliterature.Thenext

sectionwillpresentrelatedworkonwhichourcurrentworkisbased.

Waveletshavebeenextensivelyusedasacompressivedomaininthe

fieldofCS.Apartfromthepreviouslylistedtechniques;thefollowing

methodsdeservespecialattention.In[24]amethodcalledtotalvariation

compressedsensingMRI(TVCMRI)ispresented.Useoftotalvariationhelps

regularizethespatialvariabilityofthewaveletcoefficientdomain.Itusesan

operator-splittingalgorithmtosolvetheproblemofreconstructioncalled

thefastcompositesplittingalgorithm,(FCSA)[53].Thisapproachextends

equation(3)withtheadditionaltermappendedattheendofequation(4).

𝑚𝑖𝑛!{ 𝐹 𝑥 = !!𝐴𝑥− 𝑏 !

! +𝛼 𝑥 !" + 𝛽 𝜙! !}(4)

where 𝛼 𝑥 !" = (∇!"! )! + (∇!"! )!!! (5)

α=positiveparameter,wheretheterminsidetherootissummation

oftheresultoffinitedifferenceoperatorsonthefirstandsecondcoordinates

respectively,𝛽=positiveparameter, ∇denotestheforwardfinitedifference

operatorforfirstandsecondco-ordinateswhichisrepresentedintermsof𝑥

as

∇!"! = 𝑥!!!,! − 𝑥!"

∇!"!= 𝑥!,!!! − 𝑥!"

SignaltoNoiseRatiomeasuresthequalityofreconstructedimageshowin

24

equation(6).

SNR = !!

!(6)

Whereµ!=averagerootmeanofsignaland𝜎isthestandarddeviation

2.1.2 TraditionalCompressedSensingMethodologiesinMRI

Therearefewwell-knownorconventionalcompressedsensing

algorithmsthatarecommonlyusedinMRI.Someofthemarementioned

below.

1) CSofMRIusingSingularValueDecomposition(SVD)

Accordingtoauthorsof[43],thereconstructedimageMcanbe

representedusingtheSingularvalueDecompositionformula

M = UΣV!(7)

whereUandVareorthogonalmatricesandΣisdiagonalmatrixwithnon-

zerosingularvalues.

Whenexquation(7)isusedininverseFourierdomainthenweget

Ψ m = U!∗mV!(8)

whereU!andV!areleftandrightunitarymatrixthatareabletoprovidea

sparserepresentationforthedesiredimageMwhenprojectedonU!andV!

fornnumberofsamples.Forbetterresults,matricesUandVareiterated

withSVDusingtheequation(3).Figure2.2representstheoriginalimage

anditsreconstructedimageusingSVDmethod.

25

Figure2.2:RepresentstheoriginalimageanditsreconstructedimageusingSVD

method.

InFigure2.2,Thetoprowshowstheprocessofgeneratingthesparsity

basisfromlefttoright:theimageI,theunitarymatrixU,thediagonal

matrixandunitarymatrixV.ThebottomrowshowstheresultofSVDwith

sparserepresentationofdesiredimagefromlefttoright:thesparseimage

M,theunitarymatrixU,thedesiredimageMandunitarymatrixV.

2) CSofMRIusingPrincipleComponentAnalysis(PCA)

In[24]amethodcalledtotalvariationcompressedsensingMRI

(TVCMRI)ispresented.Useoftotalvariationhelpsregularizethespatial

variabilityofthewaveletcoefficientdomain.Itusesanoperator-splitting

algorithmtosolvetheproblemofreconstructioncalledthefastcomposite

splittingalgorithm(FCSA)[53].

𝑚𝑖𝑛!{F m = ∥ Φ! m − y ∥!!+ βTV(m)+ λ ∥ Ψ m ∥!}(9)

Whereβinequation(9)representstheweightofthetotalvariation(TV)thatis

theaddedtermfordenoising.

AnothermethodforCSbasedonPCAanalysisinwhichtheweightofnorm

26

LIoftheimagewasadaptivelyupdatedateachiterationwithrespectto

reductionofdimensions.In[39]method,PCAadaptiveapproachastheweight

ofsparsity,theconstrainedoptimizationisconvertedintotheminimizationof

thefollowingequation:

𝑚𝑖𝑛!{F m = ∥ Φ! m − y ∥!!+ βTV m +∥ u×s×v ∥! ∥ Ψ m ∥!}(10)

Matricesu,sandv,arethePCAelements([u,s,v)=PCA(m)).

EfficiencyandaccuracyofPCAisclaimedtobe15%higherthantheSVD

method.ResultcomparisonisshowninFigure2.3.

(a)(b)Figure2.3:(a)OriginalImage(b)CSwithPCA,redcirclemeanstheareahashigherSNR

comparedtopreviousmethods

3) CSofMRIusingWaveletTreeStructure(WaTMRI)

Waveletsarefunctionsthatsatisfycertainmathematicalrequirements

andareusedinrepresentingdataorotherfunctions.Waveletanalysisis

usedtoapproximatefunctionsthatarecontainedneatlyinfinitedomains.

Waveletsarebestsuitedforapproximatingdatawithsharpdiscontinuities.

Waveletapplicationsareusedinareassuchasimagecompression,computer

27

vision,predictionofearthquakeprediction,etc.Whenmovedtofrequency

domainfromtimedomainthenDiscreteWaveletTransform(DWT)isused

andDiscreteDaubchiesWaveletTransformisusedtoobtainwavelet

coefficients.Thewaveletdecompositionconsistsoffourcomponentscalled

Approximation,Horizontal,VerticalandDiagonaldetailsasshowninthe

Figure2.4.

(a)(b)

Figure2.4:(a)WaveletDecomposition(b)DecomposedImagedirection

ThishasfurtherbeenappliedinWaTMRI[29]inadditiontousingtotal

variation,parentchildrelationshipsofwaveletsareconsidered.Theauthors

groupallparentchildcoefficients.Theresultantoptimizationequation(11)

is

𝑚𝑖𝑛!{ 𝐹 𝑥 = !!𝐴𝑥− 𝑏 !

! +𝛼 𝑥 !" + 𝛽 𝜙! ! + (𝜙!)! !!∈! }(11)

Theadditionofthe“parent-child”group(g)termenforcestheabove

optimizationtoconsidercoefficientsinparentchildpairsandtherebyaims

todiscardfromtheresult,weakorunderperformingcoefficientgroups.

Theequation(11)issolvedusingFastIterativeShrinkageThresholding

Algorithm(FISTA)[26],[27].

28

2.1.3 Drawbacks

AlthoughtheWaTMRIapproachconsiderscorrelationamongthe

waveletcoefficients,itisrudimentary.Italsosuffersfromthedisadvantages

ofconventionalwavelettransform:sensitivetoshifting[31],poor

directionality[32]anddearthofphaseinformation[33].

2.1.4 ProblemStatement

Problemtoberesolvedisimagereconstructionwithminimalerroras

wellastospeeduptheruntimeofthealgorithm.Thedirectionalityof

waveletistakenintoconsideration.Sincethetargetistogeneralizethe

algorithmtocatertogeneralcaseofimagereconstructionthisisachallenge

initselfsincetherearetraditionalimageprocessingalgorithmsthatworks

quietefficientlyaswell.ConsiderationofNon-MRIdatatypeprovidesa

generalizationofdifferentdatadistributedthatcouldbeusedtosimulate

otherMRIdatasetthatarehardtoobtain.

29

Chapter3

3.1 ProposedSolutions

Inthispaper,anewmodelisproposedwithwavelettreesparsitytaken

intoconsiderationalongwithentirewavelet“group-coefficients”.Thetree

structureshowninFigure3.1,ismodeledasanoverlappinggroup

regularization,withgroupssharingGrandparent(GP)->Parent(P)->Child(C)

coefficients.Theoriginalproblemof20%samplingratecanbedecomposedto

two-subproblemsandsolvedveryefficientlybyusingtheFISTA[26],[27].

Equation(12)presentsourapproachintermsoftheoptimizationterms.

min!{ F x = !!Ax − b !

! + α x !" + β( ϕx ! + (ϕ!)! !)}!∈! , s. t. t > τ(12)

whereτisprescribedco-efficienttreethresholdforeachlevelofhierarchy.

Subsett ∈TiswhereTisthenumberofhierarchicaltreegroups.

Theequation(12)hasthreeterms,whichspecifythethreecomponentsofthe

algorithm.Thefirsttermensuresreconstructionerroriswithinrange.The

secondtermsenforcessmoothnessontheresult.Thefinaltermenforcessparse

selectiononlyfromthegroupsoftreecoefficientgreaterthanprescribed

thresholdvalue.

30

(a)(b)Figure3.1:WaveletQuad-TreeStructure(a)AcardiacMRimage(b)Corresponding

WaveletTreeGroupofHierarchyGP-P-CStructure

Weproposedthreenewalgorithms,whichwouldexploitthehierarchical

approach.

• WeightedThresholdWaveletTreeStructure(WTWTS)

• DirectionalWaveletTreeStructure(DWTS)

• WeightedDirectionalWaveletTreeStructure(WDWTS)

3.1.1 WeightedThresholdWaveletTreeStructure(WTWTS)

Thehierarchy-basedapproachiscombinedwiththeweighted

thresholdingtoprovideabaselineforthisresearch.Theothertwoalgorithms

arebasedonthisconceptandbuildonitwithdifferenttechniquesusedinthe

shrinkageofembeddedzerotreealgorithm( z!)asshowninequation(13)

whichisconsideredinpseudocodeinTable3.1.

z! = shrinktree((Tϕ!)!,!!, τ)(13)

wherethez!canbefurthersimplifiedto

z! = max ((Tϕ!)! −!!, 0)

31

andwhereϕ! ≥ τandϕ! < τforz!=0suchthatthasthefollowingthreshold

range

Child (C) = 25% < τ ≤ 50%ofCco-efficient

Parent (P) = 25% < τ ≤ 75%ofPco-efficient

GrandParent(GP) = 75%ofGPco-efficient

Table3.1:PseudoCodeofWTWTS

Input:𝑟!=𝑥!!,𝑡!=1,α,β,λ, 𝜏

forn=1toMaxIteration=50do

1. Calculate𝑧! =𝑠ℎ𝑟𝑖𝑛𝑘𝑡𝑟𝑒𝑒 ((𝑇𝜙!)! ,!!, 𝜏)

2. 𝑥! = 𝑟! − 𝑎𝑟𝑔𝑚𝑖𝑛!!{!!𝐴𝑟! − 𝑏 !

! + !!∥ 𝑧! − (𝑇𝜙!)!! ∥}

3. 𝑥! = 𝑔! 𝑥 + 𝛼 ∥ 𝑥!" − 𝑥! ∥!!

4. 𝑥! = 𝑔! 𝑥 + 𝛽 ∥ 𝜙𝑥 − 𝑥! ∥!!

5. 𝑥! = (!!!!!)!

6. 𝑡!!! = [!! !!! !! !

!

7. 𝑟! + 1 = 𝑥! + !!!!

!!!!(𝑥! − 𝑥!!!)

wherehyper-parametersandparametersareexplainedbelow:

α=0.001

β=0.035=positiveparameter

λ=0.2xβ=0.007=multiplier(usedinallconvexmodels)

n=iterationnumber

τ=Weightedthresholdconditionmentionedbelow




(Tϕ!)!=Non-overlappinghierarchicaltreeinsparsedomain

32

r!=x!!eachsignalcoefficientvaluefrombasediteration

g! x =α x !"

g! x =β ϕ! !

t!=1,α,β,λ, τallthevaluewhichisinitializedforthefirstiterationandlater

onbasedonthethresholdvaluetheyareiterativelyupdatedfornsamples.

Thesteps5,6,7intheTable3.1arereferredfromWatMRI[29].Resultsshow

theefficacyofourresentedapproach.

3.1.2 DirectionalWaveletTreeStructure(DWTS)

Thesecondapproachworthhighlightinginthispaperistheconceptof

waveletdirectionalityasexpressedinDirectionalFilterBanksTransform

(DFB)[33]asshowninFigure3.2andFigure3.3.TheDirectionalfilter

banksarerealizedusingiteratedquincunxfilterbanks.Thedirectional

filterbankconceptisextendedtomulti-resolutionwaveletsubband

decompositionasbyaddingthefeatureofdirectionalityinanappropriate

mannerimprovementmayoccurinthenonlinearapproximationof

waveletsasshowninFigure3.4.

Figure3.2:Directionalfilterbankfrequencypartitioningusingeightdirections

(a)(b)

Figure3.3:(a)Verticaldirectionalfilterbanks(b)Horizontaldirectionalfilterbanks

33

Figure3.4:ExampleofDirectionalfilterbankonrepresentation

Thesecondmethodutilizesthepriordiscusseddirectionalwavelet

transforms.Directionalityincreasestheefficiencyofthewavelettransforms.

Hencewavelettransformscanbereplacedwithdirectionalwavelets.This

enablesbetterconsolidationofthe‘wavelettrees’andtherebyenablesbetter

sparseoptimizedimagereconstruction.Figure3.5showstheeightdirectional

filterbankonanimage.

Figure3.5:EightDirectionalfilterbanksrepresentusingclassicBarbaraImage

34

ThealgorithmissimilartotheproposedalgorithmWTWTSusingHybrid

WaveletsandDirectionalFilterBanks[33],asbothusethresholdingtoreduce

thepotentialnumberof“candidategroupofTrees”forsparseprocessing.The

reductionofcandidategroups,bythresholdinggroupswithlowscores,enables

sparseselectionfromrichlypopulatedgroups.Shrinkagealgorithmusesthe

eightDirectionalfilterbankwaveletasshowninTable3.2.

Table3.2:PseudoCodeofDWTS




2. 𝑥! = 𝑟! − 𝑎𝑟𝑔𝑚𝑖𝑛!!{!!𝐴𝑟! − 𝑏 !

! + !!∥ 𝑧! − (𝑇𝜙!)!! ∥}

3. 𝑥! = 𝑔! 𝑥 + 𝛼 ∥ 𝑥!" − 𝑥! ∥!!

4. 𝑥! = 𝑔! 𝑥 + 𝛽 ∥ 𝜙𝑥 − 𝑥! ∥!!

5. 𝑥! = (!!!!!)!

6. 𝑡!!! = [!! !!! !! !

!

7. 𝑟! + 1 = 𝑥! + !!!!

!!!!(𝑥! − 𝑥!!!)

whereDirectionalWaveletisconsidered.

Hyper-parametersandparametersareexplainedbelow:

α=0.001


λ=0.2xβ=0.007=multiplier(usedinallconvexmodels)

n=iterationnumber



35





g! x =α x !"

g! x =β ϕ! !

t!=1,α,β,λ, τallthevaluewhichisinitializedforthefirstiterationandlater


Thesteps5,6,7intheTable3.2arereferredfromWatMRI[29].Resultsshow

theefficacyofourresentedapproach.

3.1.3 WeightedDirectionalWaveletTreeStructure(WDWTS)

Inthefinalapproach,inadditiontothedirectionalwavelet,weenforce

thresholdconstraintonthedirectionality.Inadditiontowaveletthresholding,

newestalgorithmalsoincorporatesdirectionalwaveletthresholding(DWTS).

Asseeninthelasttermofequation(12),boththresholdingofwavelet

coefficientgroup’sdirectionalityandenergyareconsidered.Theshrinkage

algorithminequation(13)ismodifiedwithweighteddirectionalcondition.

Table3.3showsthepseudocodeofWDWTSalongwiththeconditionbelow:

36

Table3.3:PseudoCodeofWDWTS




2. 𝑥! = 𝑟! − 𝑎𝑟𝑔𝑚𝑖𝑛!!{!!𝐴𝑟! − 𝑏 !

! + !!∥ 𝑧! − (𝑇𝜙!)!! ∥}

3. 𝑥! = 𝑔! 𝑥 + 𝛼 ∥ 𝑥!" − 𝑥! ∥!!

4. 𝑥! = 𝑔! 𝑥 + 𝛽 ∥ 𝜙𝑥 − 𝑥! ∥!!

5. 𝑥! = (!!!!!)!

6. 𝑡!!! = [!! !!! !! !

!

7. 𝑟! + 1 = 𝑥! + !!!!

!!!!(𝑥! − 𝑥!!!)

wherehyper-parametersandparametersareexplainedbelow:

α=0.001


𝜆=0.2xβ=0.007=multiplier(usedinallconvexmodels)

n=iterationnumber


s. t. t 2,3,6,7 Inner Direction = 75% of co− efficient1,4,5,8 Outer Direction = 25% of co− efficient P,C

P=Parent

C=Child



g! x =α x !"

g! x =β ϕ! !

𝑡!=1,α,β,λ, 𝜏allthevaluewhichisinitializedforthefirstiterationandlater


37

3.2 ComplexityAnalysis

Supposewerepresentxasanimagewithnpixels,overallcomplexity

wouldbeO(n’+nlogn)wheren’isnumberofnon-zerohierarchicaltree

element.SothatTisnon-overlappinghierarchicaltreewouldhaveO(n’).

BelowisacomplexityofeachstepinthreealgorithmnamelyWTWTS,DWTS,

WDWTS.

Step1:O(n’+nlogn)

Step1showsthecomplexityforWTWTS,DWTSandWDTWTS.Notethat

weconsidered8directionsforDWTS,WDWTSitwouldbe

O(n’+8nlogn)≈O(n’+nlogn).

Step2:O(nlogn)

Step3:O(n)

Step4:O(nlogn)

Step5:O(n)

Step6:O(n)

Step7:O(n)

Resultsinthenextsectioncomparethethreeproposedtechniquesandthe

currentbenchmarktechnique.

38

Chapter4

4.1 Experimentsandresults

Numerousvarietiesofexperimentshavebeenconductedfordifferent

parametersandhyperparametersselection.Thissectionalsoshowsfinalresult

oftheproposedalgorithmsperformancecomparedwiththeWatMRI[29].Inthe

MRimagingproblem,AisthepartialFouriertransformwithmrowsandn

columns.Thisstudyconsidered20%sampling.MRscanningtimewillbe

reducediflessmeasurementissampled.Allmeasurementscontain0.01Gaussian

WhiteNoise.SignalToNoiseRatio(SNR)andMeanSquareError(MSE)are

consideredforevaluation.

4.1.1 TestEnvironmentSpecification

Detaileddescriptionofthetestenvironmentthatwasusedforthethesisis

mentionedbelow:

• HardwareSpecification:

o System:MacOSv10.13.1

o Processor:2.4GHzIntelCorei5

o Memory:8GB1333MHzDDR3(DoubleDataRateSynchronous

DynamicRandom-AccessMemory)

o Graphics:IntelHDGraphics3000512MB

• ProgrammingLanguage:MATLABR2017a(Aproprietaryprogramming

languagedevelopedbyMathWorks)

4.1.2 DataSet

BothMRIandNon-MRIdatatypesareconsideredtovalidatethis

39

approach.500setsofeachdatatypeswithdifferentsizefrom16x16,32x32,

64x64,128x128,and256x256resolutionsareconsidered.

MRIData:

MRIdatawasobtainedfromthereferencepaper[29].Brain,heart,

shoulder,chestimagesareconsideredasshownintheFigure4.1.

Figure4.1:MRIData

ThesamplingmaskthatisappliedforspatialfilterisasshowninFigure4.2.

Figure4.2:SamplingMask

ThetwobasictypesofMRIimagesareT1-weightedandT2-weightedimages,

oftenreferredtoasT1andT2images.

T1images–highlightfattissuewithinthebodyasshowninFigure4.3(a)

T2images–highlightfattissueandwatercontentwithinthebodyasshownin

Figure4.3(b)

40

(a)(b)

Figure4.3:(a)T1imageand(b)T2image

Experimentsalsoconsidered50imagesofT1,T2,whichrepresentdifferent

detailsinthebraindataoftoidentifyifthereisanyvariationbasedonthe

directionofthedata.Thedatasetwasobtainedfromgithublinkasshownbelow:

https://github.com/muschellij2/Neurohacking_data/tree/master/BRAINIX/DIC

OM

NonMRIData:

Non-MRIdatatypeswerepickedupfromGoogleImagebasedonfollowing

criteriasuchas

• PortraitImages–Thisdatatyperepresentsimagesthathaveboth

combinationsoflowandhighpasssignalsthatmeantheonlyfewofthe

datacanbesparselyrepresented.FilesarenamedasLena,Obama,

Einstein,andCameramanthatisJPGfilesasshowninFigure4.4.

Figure4.4:PortraitData

41

• CrowdImages–Thisdatatyperepresentsimagesthathavemajorlyhigh

passsignalthatmeanstherearelotsofvariationsinthedata.Filesare

namedasCrowd1,Crowd2,Crowd3andCrowd-riceimagesthatareJPG

filesasshowninFigure4.5.

Figure4.5:CrowdData

• LandscapeImages–Thisdatatyperepresentsimagesthathavemajority

ofthelowpasssignalsthatindicatesthatsomeofthedataareredundant

whichcouldbesparselyrepresented.FilesarenamedasLandscape1,

Landscape2,andLandscape3imagesasshowninFigure4.6.

Figure4.6:LandscapeData

4.1.3 Hyper-Parameters

Eachsimulationonthedataiscarriedoutwith50iterationswithapproximate

20%sampling.WaveletTreeStructureisconfiguredtochoosetheGrandparent,

42

parentandchildrelationshipthataregeneralizedforCompressedSensingin

MRI.Differentthresholdingsetisalsoconsideredashyper-parametersthatare

configured.

4.1.4 Parameters

WaveletTreeGroupingiscustommadebasedonwhichofthewaveletco-

efficientcontributestowardsaccurateimagereconstruction.WaveletCo-

efficientchangesarebasedonthethresholdingsetsthatneedtobeiteratively

updatedtoprovideco-efficientstoimprovethereconstructionoffinalimages.

4.1.5 PreliminaryResults

4.1.5.1 SelectionofThresholdset,ImagesizeandthresholdbasedonWaveletDirections

Numberofzero-tree(zerovalueco-efficientatalllevel)variedagainst

rangeofthreshold,therewasnoparticularonevalue.Ithasbeendifferentfor

eachdataofMRIandnon-MRIdata.

Figures4.7to4.10showsvariationsofthenumberofzerotrees

occurrence(eliminatedduringshrinkage)ateachhierarchylevelwithdifferent

setsofdatadisplaysaspecificrangeofthresholdτwhichnormallyisbetween

0-100.

43

Figure4.7:VariationofnumberofzerotreesbasedonGP-P-CHierarchyforonebrain

image

Figure4.8:VariationofnumberofzerotreebasedonGP-P-CHierarchyforT1images

44

Figure4.9:VariationofnumberofzerotreesbasedonGP-P-ChierarchyforT2image

Figure4.10:VariationofnumberofzerotreesbasedonGP-P-Chierarchylevelforentire

500brainimages

45

Thevariationsofnumberofzerotreesoccurrenceinthedatawereattributed

tovariationintheimage.Tocomparetheentropydistributionoftheimage

basedonthelevelwouldgiveabetterunderstandofdistributionpatternin

eachlevelirrespectiveofthetypeofimagesasshownintheFigure4.11.

Figure4.11:3DGraphofrangeofthresholdsetsagainstEntropyofdataatParent-Child

Level

TheParentandChildleveldistributionledtousingtheconceptsofweighted

thresholdingsincethereisnoonethresholdvaluethatsatisfiesallthedata

type.Thereshouldbedifferentrangeofthresholdsettocatertodifferent

distributioninlevel.Henceweightedtreestructureofdifferentcombinationof

ratioofthresholdisbeingconsidered.(GP,P,C)thresholdinpercentagesare:

• 75,75,50–Set1

• 75,75,25–Set2

• 75,50,50–Set3

• 75,50,25–Set4

46

• 75,25,25–Set5

Figures4.12to4.15showscomparisonofthe5setsofthresholdvaluesagainst5

setsofdifferentresolutionsize(16,32,64,128,256).

Figure4.12:ComparisonofdifferentthresholdingsetsagainstSNRfordifferentresolution

sizesforBrainImages(RedlineindicatereferenceSNRofWatMRIalgorithm)


sizesforHeartImages(RedlineindicatereferenceSNRofWatMRIalgorithm)


sizesofShoulderImages(RedlineindicatereferenceSNRofWatMRIalgorithm)

47


sizesforChestImages(RedlineindicatereferenceSNRofWatMRIalgorithm)

Itwasfoundthat256x256wasthemoststablecomparedtoothers.Inregards

tothresholding,itwasfoundthatSet2and3gavethebestandmostreliable

resultswithhigherSNRvaluecomparedtoWatMRIasshownintheFigure

4.16.

Figure4.16:SNRcomparisonbetweenWatMRI(Greencolor)andWTWTSwithSet2(Red

color)and3(Bluecolor)approaches

48

Figure4.17showsthebehaviorofnon-MRIdataagainstthethresholdingset2

and3whencomparedtothereferencealgorithm.

Figure4.17:ComparisontheSNRforPortrait,CrowdandLandscapeimagesforthresholdingset2and3(RedlineindicatesreferenceSNRofWatMRIalgorithm)

Figures4.18to4.21showsthevariationofhybridwaveletcoefficientsinthe8

directionsforeachMRIdata.Direction2,3,6,and7thatisconsideredasinner

directionhashighercoefficientsthandirection1,4,5and8thatisconsideredas

outerdirection.ThisistheconditionforthealgorithmWDWTStochoosethe

weighteddirectionalthresholdingintheParentandChildlevelwhileallvaluesof

GrandParentlevelwillbeconsideredsinceallthecoefficientscontributetowards

theimagereconstruction.

Figure4.18:Histogramof8DirectionsatParentandChildlevelforBrainImage

49

Figure4.19:Histogramof8DirectionsatParentandChildlevelforHeartImage

Figure4.20:Histogramof8DirectionsatParentandChildlevelforShoulderImage

50

Figure4.21:Histogramof8DirectionsatParentandChildlevelforChestImage

Aboveresultsshowsthatexperimentswerecarriedouttoselectthresholdsets,

imagesizeandthresholdfordirectionalwaveletonMRIdata.Theseexperiments

werecarriedoutnon-MRIdataobtaininggoodresultthatcouldbegeneralized

foranytypeofimages.Figures4.22to4.24showthePortrait,Crowdand

Landscapeimagesresultsbasedontheselectionmadeforthresholdset,image

sizeandthresholdonwaveletdirection.

51

Figure4.22:ComparisonofPortraitimageagainstoriginaldatausingWTWTSand

WDWTSalgorithmswithchosenthresholdset

Figure4.23:ComparisonofCrowdimageagainstoriginaldatausingWTWTSandWDWTSalgorithmswithchosenthresholdset

52

Figure4.24:ComparisonofLandscapeimageagainstoriginaldatausingWTWTSandWDWTSalgorithmswithchosenthresholdset

4.1.6 ResultsandEvaluation

Thissectiondisplaystheevaluationofproposedalgorithmsincorporating

thefindingofhyper-parameterssuchasimagesize,thresholdsets,thresholdfo

weighteddirectionetc.

Figures4.25to4.31belowshowtheresultcomparisonofthethreenew

algorithmsdevelopedasthepartofthisthesisagainstreferencepaperWatMRI.

EachdatatypehasitsSignaltoNoiseRatio(SNR)valuesandMeanSquare

Error(MSE)

53

Figure4.25:Brainimagereconstructionusingthe3algorithmscomparedagainstWatMRI

Figure4.26:Heartimagereconstructionusingthe3algorithmscomparedagainstWatMRI

54

Figure4.27:Shoulderimagereconstructionusingthe3algorithmscomparedagainst

WatMRI

Figure4.28:Chestimagereconstructionusingthe3algorithmscomparedagainstWatMRI

55

Figure4.29:Portraitimagereconstructionusingthe3algorithmscomparedagainst

WatMRI

Figure4.30:Crowdimagereconstructionusingthe3algorithmscomparedagainst

WatMRI

56

Figure4.31:Landscapeimagereconstructionusingthe3algorithmscomparedagainst

WatMRI

Table4.1showstheSNRandMSEcomparisonofMRIdataforallthethree

algorithmsagainstWatMRI.

Table4.1:SNRandMSEcomparisononMRIdata

Method

MRIDataTypes

Brain Heart Shoulder Chest

SNR MSE SNR MSE SNR MSE SNR MSE

WatMRI 19.73 28.74 20.80 16.32 23.61 6.36 20.39 63.68

WTWTS 19.98 27.16 20.87 16.07 23.98 5.85 20.41 63.35

DWTS 20.54 23.86 21.47 13.99 23.99 5.80 20.44 62.97

WDWTS 20.83 22.33 21.69 13.32 25.07 4.54 21.06 54.60

57

Table4.2showtheCPUtimecomparisononMRIdatasetsforallthethree


Table4.2:CPUTimecomparisononMRIdata

Method

MRIDataTypes

Brain Heart Shoulder Chest

CPUTime(s) CPUTime(s) CPUTime(s) CPUTime(s)

WatMRI 1.50±0.03 1.53±0.05 1.60±0.04 1.59±0.03

WTWTS 1.38±0.09 1.35±0.1 1.43±0.11 1.41±0.09

DWTS 1.40±0.08 1.43±0.09 1.50±0.07 1.47±0.05

WDWTS 1.42±0.08 1.66±0.05 1.70±0.6 1.51±0.11

Table4.3showtheSNRandMSEcomparisononNon-MRIdatasetsforallthe

threealgorithmsagainstWatMRI.

Table4.3:SNRandMSEcomparisononNon-MRIdata

Method

NonMRIDataTypes

Portrait Crowd Landscape

SNR MSE SNR MSE SNR MSE

WatMRI 23.40 80.10 14.26 420.3 19.81 104.05

WTWTS 24.19 66.90 14.46 401.56 20.11 97.09

DWTS 24.37 64.15 14.46 401.07 20.15 96.14

WDWTS 24.67 59.88 14.70 379.26 20.20 94.98

58

Table4.4showtheCPUtimecomparisononNon-MRIdatasetsforallthethree


Table4.4:CPUTimecomparisononNon-MRIdata

Method

Non-MRIDataTypes

Portrait Crowd Landscape

CPUTime(s) CPUTime(s) CPUTime(s)

WatMRI 1.50±0.01 1.60±0.03 1.55±0.04

WTWTS 1.45±0.09 1.50±0.05 1.42±0.08

DWTS 1.49±0.07 1.53±0.06 1.47±0.04

WDWTS 1.55±0.12 1.61±0.05 1.49±0.08

4.1.7 ComparisonbetweenWTWTS,DWTS,WDWTS

1. AllthreealgorithmsnamelyWTWTS,DWTS,WDWTShaveshownbetter

qualityofreconstructedimagerepresentedbySNRvalueandreductionin

MSEvaluethanthereferencepaper(WatMRI[29])result.

2. WTWTSproduced≈4%imagequalitywith≈8%reductioninalgorithm

runthetimeofthealgorithm.(≈symbolrepresentsapproximation)

3. DWTSproduced≈10%qualitywithreductionof≈4%inruntime.

4. DWTSalsoconsumedmorememoryspaceforstoringdirectional

informationinthetreestructure

5. WDWTSproduced≈20%betterimagequalitywith≈6%reductioninrun

timeofthealgorithm.

6. TradeoffofWDWTSisthememoryspaceconsumptionforweighted

directionaltreestructure.

59

Chapter5

FutureWork

Thisresearchcanbetakenforwardtoimprovetheresults.Ideasareasfollows:

• UseDeepNeuralNetworktotrainmachinetolearnthealgorithm

withoutanyassistancefromradiologistsandtrainthereconstruction

errorbackintothenetworktominimizetheerror.Theoverviewisas

shownFigure5.1.

Figure5.1:OverviewofDeepNeuralNetworkforCSinMRI

TrainReconstructionError:Flowchartofthetrainingreconstructionerroris

showninFigure5.2.

60

Figure5.2:ReconstructionErrorflowchart

• UpdatealgorithmforMulti-slice3DMRI–MultiplesliceofDataforone

directionalimagecanbeconsideredoreven3DMRIdatacanbe

consideredasshowninFigure5.3.Thethreecrosssectionaldirectional

ofthebrainimageisrepresentedinFigure5.4.

Figure5.3:Multi-SliceofBrainMRI[38]

ReconstructionErrors

TrainNeuralNetwork

MinimizeReconstruction

Errors

61

Figure5.4:3DBrainMRIdisplayingeachdimension[40]

DynamicMRIwithTensorwillbeusedtoworkon3DMRIdataset.Eachofthe

dimensionsisconsideredtobe2Ddatasetalongwithtime.Figure5.5shows

theoverviewof3DMRItoTensor(3Directionalaxisorequivalentofvolume

datastructure)[37,38].

Figure5.5:DynamicMRIrepresentedinTensor[46]

62

TheformulaforTensorissimilartoCSwithSVDbutthedatais3D[45].Sothe

needistoconsiderHigherOrderSingleValueDecomposition(HOSVD)as

showninequationbelow

𝑀 = 𝑈𝑆𝑉! = 𝑆 ×!𝑈!×!𝑉! = 𝑆 ×!𝑈!×!𝑈!(14)

whereU&V=unitarymatrices,whileS=pseudo-diagonalmatrixwhichis

representinequation(15)

𝑆 = 𝐴!×!𝑈!!×!𝑈!!×!…… .×!𝑈!! (15)

whereA0=nthordertensorunfoldedmatrixasshownbelowinequation(16):

𝑖!!! − 1 𝐼!!!𝐼!!!…… . . 𝐼!𝐼!𝐼!…… 𝐼!!! +

𝑖!!! − 1 𝐼!!!𝐼!!!…… . . 𝐼!𝐼!…… 𝐼!!! +⋯+

𝑖! − 1 𝐼!𝐼!…… . . 𝐼!!! + 𝑖! − 1 𝐼!𝐼!… . . 𝐼!!! +⋯+ 𝑖!!!(16)

Figure5.6representsthe3Dtensorand4DtensorusedforSVDcalculation.

Thisiscommonlyusedink-tFOCUSS[42,43]methodinmedicalcommunity.

Figure5.6:VisualizationofHigherOrderSingleValueDecomposition(HOSVD)(a)3D

Tensor(b)4DTensor[45]

63

Chapter6

Conclusion

TheclearimpetusofchoosingCompressiveSensing(CS)approachisto

utilizesignificantlesssamplingwhilegeneratingahighqualityreconstruction.

RecentmethodscanreconstructMRimageswithgoodqualityfrom

approximate20%sampling.Theseresultshavealsoshownsuchtrendsaswell.

Theproposedmethodsshowimprovementoverthecurrentbenchmark

algorithminthisfield.Althoughsomealgorithmshavebeenproposedto

improvestandardCSrecoverybyutilizingthetreestructureofwavelet

coefficients[10][11][12][13],noneofthemexploitthetreesparsitytoany

significantinstantandonlyafewofthemconductexperimentsonMRimagesto

validatethepracticalbenefitforacceleratedMRI[51][52].

ThisresearchhasfuturefocustobecombinedwithDeepNeuralNetwork

andMulti-slicingof3DMRItothatwouldincorporatemulti-dimensionaldata

intoCompressedSensingforMRI.Thissurelywouldcontributetotheexisting

research.

64

Bibliography[1] W.C.Shiel,“MRIscan”,https://www.medicinenet.com/mri_scan

[2] FigureisadaptedfromSingaporeSportsandOrthopedicClinic.Link:

https://www.orthopaedics.com.sg/treatments/other-services/magnetic-

resonance-imaging-mri/

[3] M.A.Qureshi,M.Deriche,“Anewwaveletbasedefficientimage

compressionalgorithmusingcompressivesensing”,articleinMutlimedia

ToolsandApplication,2015.

[4] H.Nyquist,"Certaintopicsintelegraphtransmissiontheory".Trans.

AIEE.47:617–644.(April1928)

[5] C.E.Shannon,"Communicationinthepresenceofnoise".Proceedingsof

theInstituteofRadioEngineers.37(1):10–21.(January1949)

[6] C.E.Shannon,"AMathematicalTheoryofCommunication".BellSystem

TechnicalJournal.27(4):623–666.(October1948).

[7] E.T.Whittaker,"OntheFunctionswhicharerepresentedbytheexpansions

oftheInterpolationTheory".Proc.RoyalSoc.Edinburgh.35:181–

194.(1915).

[8] R.G.Baraniuk(2007)Compressivesensing[lecturenotes].Signal

ProcessingMagazine,IEEE24(4):118–121

[9] R.G.Baraniuk,E.Candes,M.Elad,Y.Ma(2010)“Applicationsofsparse

representationandcompressivesensing“.ProceedingsoftheIEEE98(6):

906–909

[10] E.Candes,T.Tao,“Near-optimalsignalrecoveryfromrandomprojections:

Universalencodingstrategies?”IEEETransactionsonInformationTheory

52(12):5406–5425.(2006)

65

[11] E.Candes,M.B.Wakin,“Anintroductiontocompressivesampling”.Signal

ProcessingMagazine,IEEE25(2):21–30.(2008)

[12] E.Candes,J.Romberg,T.Tao,“Robustuncertaintyprinciples:Exactsignal

reconstructionfromhighlyincompletefrequencyinformation”.IEEE

TransactionsonInformationTheory52(2):489–509.(2006)

[13] E.Candes,J.Romberg,T.Tao,“Stablesignalrecoveryfromincompleteand

inaccuratemeasurements.”Communicationsonpureandapplied

mathematics59(8):1207–1223.(2006)

[14] D.L.DonohoD.L.Compressedsensing.IEEETransactionsonInformation

Theory52(4):1289–1306.(2006)

[15] E.Candes,J.RombergandT.Tao,“Robustuncertaintyprincipals:Exact

signalreconstructionfromhighlyincompletefrequencyinformation,”IEEE

Trans.Inf.Theory,vol.52,pp.489-509,2005.

[16] E.Candes,T.Tao,“Decodingbylinearprogramming.”IEEETransactionson

InformationTheory51(12):4203–4215.(2005)

[17] E.Candes,J.Romberg,“Sparsityandincoherenceincompressivesampling.

Inverseproblems”.23(3):969.(2007)

[18] G.H.GolubandD.P.O’Leary,Somehistoryoftheconjugategradient

methodsandLanczosalgorithms:1948-1976,SIAMRev.,31(1989),pp.

50–100.

[19] M.Lustig,D.Donoho,J.M.SantosandJ.M.Pauly,“Compressedsensing

MRI,”IEEESignalProcess.Mag.,vol.25,no.72,pp.72-82,2008.

[20] M.Lustig,D.Donoho,andJ.M.Pauly,“SparseMRI:Theapplicationof

compressedsensingforMRimaging”,MagneticResonanceinMedicine,vol.

58,no.6,pp.1182-1195,2007.

[21] T.CaiandL.Wang,“OrthogonalMatchingPursuitforSparseSignal

RecoveryWithNoise”,IEEETransactionsonInformationTheory,Volume:

57,Issue:7,July2011

[22] S.MallatandZ.Zhang.MatchingPursuitwithTimeFrequencyDictionaries.

IEEETrans.Sign.Proc.(1993).

66

[23] J.Yang,Y.ZhangandW.Yin,“Afastalternatingdirectionmethodfortvl1-l2

signalreconstructionfrompartialFourierdata,”IEEEJ.Sel.TopicsSignal

Process.vol.4,no.2,pp.288-297,2010.

[24] S.Ma,W.Yin,Y.Zhang,andA.Chakraborty,“Anefficientalgorithmfor

compressedMRimagingusingtotalvariationandwavelets,”Proceedingof

IEEEConferenceonComputerVisionandPatternRecognition,pp.1-8,

2008.

[25] J.Huang,S.ZhangandD.Metaxas,“EfficientMRimagereconstructionfor

compressedMRimaging,”MedicalImageAnalysis,vol.15,no.5,pp.670-

679,2011.

[26] A.BeckandM.Teboulle,“Afastiterativeshrinkage-thresholdingalgorithm

forlinearinverseproblems,”SIAMJ.onImagingsciences,vol.2,no.1,pp.

183-202,2009.

[27] A.BeckandM.Teboulle,“Fastgradient-basedalgorithmsforconstrained

totalvariationimagedenoisinganddeblurringproblems,”IEEETrans.on

ImageProcess.,vol.18,no.113,pp.2419-2434,2009.

[28] C.ChenandJ.Huang,“ThebenefitoftreesparsityinacceleratedMRI”,

MICCAIworkshop2012.

[29] C.ChenandJ.Huang“CompressivesensingMRIwithwavelettree

sparsity,”proceedingofthe26thAnnualConferenceonNeuralInformation

ProcessingSystems,pp.1115-1123,2012.

[30] J.Huang,T.ZhangandD.Metaxas,“Learningwithstructuredsparsity,”J.of

MachineLearningResearch,vol.12,pp.3371-3412,2011.

[31] R.G.Baraniuk,V.Cevher,M.F.DuarteandC.Hegde,“Model-Based

compressivesensing,”IEEETrans.onInf.Theory,vol.56,no.4,pp.1982-

2001,2010.

[32] F.Back,R.Jenatton,andG.Obozinski,“Structuredsparsitythroughconvex

optimization,”TechnicalReport,HAL00621245-v2,toappearonstatistical

science,2012.

[33] R.EslamiandH.Radha,“NewImageTransformsUsingHybridWavelets

67

andDirectionalFilterBanks:AnalysisandDesign”,ICIP,2005.

[34] Chen,J.Huang,“Exploitingthewaveletstructureincompressedsensing

MRI”,MagneticResonanceImaging32(2014)1377–1389.

[35] N.Kingsbury,“Imageprocessingwithcomplexwavelets,”Philosophical

TransactionsoftheRoyalSocietyofLondonA,no.357,pp.2543–2560,

1999.

[36] F.Fernandes,“Directional,shift-insensitive,complexwavelettransforms

withcontrollableredundancy,”Ph.D.thesis,RiceUniversity,2002.

[37] Y.Kim,M.Altback,T.Trouard,andA.Biligin,“Compressedsensingusing

dualtreecomplexwavelettransform”ProceedingoftheInternational

SocietyforMagneticResonanceinMedicine,p.2814,2009.

[38] VRunge,JRichter,TBeck,MPiccirelli,AValavanis,“SimultaneousMulti-

Slice(SliceAccelerated)InstituteforDiagnosticandInterventional

RadiologyClinicsforNeuroradiologyandNuclearMedicineUniversity

HospitalZurich.

[39] J.Zamani,A.N.Moghaddam,H.S.Rad,“MRIReconstructionThrough

CompressedSensingUsingPrincipleComponentAnalysis(PCA)”,ICEEMay

2012.

[40] ImagefromHitachiproductgeneratedaNeurologicalMRIusingisoFSE.

[41] DiagramfromMathiasBlasche,ChristophForman,SiemensMAGNETOM

Flash,2016.

[42] H.Jung,J.C.Ye,E.Y.Kim,(2007),“Improvedk-tBLASTandk-tSENSEusing

FOCUSS”.Physicsinmedicineandbiology52:3201–3226.

[43] H.Jung,K.Sung,K.SNayak,E.Y.Kim,J.C.Ye,(2009),“k-tFOCUSS:Ageneral

compressedsensingframeworkforhighresolutiondynamicMRI”.

MagneticResonanceinMedicine61:103–116.

[44] M.Hong,Y.Yu,H.Wang,F.Liu,andS.Crozier,"CompressedsensingMRI

withsingularvaluedecomposition-basedsparsitybasis,"Physicsin

MedicineandBiology,vol.56,p.6311,2011.

[45] Y.Yeyang,J.Jin,L.Feng,S.Crozier,MultidimensionalCompressedSensing

68

MRIUsingTensorDecomposition-BasedSparsifyingTransform”,PLosone

Volume9,Issue6,e98441.(June2017)

[46] Y.Xiaomei,L.Yuewan,C.Siji,Z.Xiujuan,Y.Qin,L.Kai,“DynamicMRI

reconstructionfromhighlyundersampled(k,t)-spacedatausingweighted

Schattenp-normregularizeroftensor”,MagneticResonanceImaging37

260–272.(2017).

[47] https://www.radiologymasterclass.co.uk/tutorials/mri/t1_and_t2_images

[48] http://multiple-sclerosis-research.blogspot.com/2015/01/the-only-thing-

that-is-black-and-white.html

[49] https://github.com/muschellij2/Neurohacking_data/tree/master/BRAINI

X/DICOM

[50] M.H.Conde,“FundamentalsofCompressiveSensing”,Chapter3from

CompressiveSensingforthePhotonicMixerDevice,DOI:10.1007/978-3-

658-18057-7_3

[51] ThisthesiswaspresentedatGeorgiaCouncilofGraduateSchools(GCGS

April2018)asapartof3MinuteThesis

[52] S.S.JanneyandS.Chakravarty,“NewAlgorithmforCS-MRI:WTWTS,

DWTS,WDWTS”,ResearchPosterpaperacceptedby40thAnnual

InternationalIEEEEMBSConferenceJuly2018

[53] M.F.Jiang,J.Jin,F.Liu,“Sparsity-constrainedSENSEreconstruction:an

efficientimplementationusingafastcompositesplittingalgorithm”

MagneticResonanceImaging.2013;31(7):1218–1227.doi:

10.1016/j.mri.2012.12.003.

[54] J.Huang,S.Zhang,H.Li,D.Metaxas,“Compositesplittingalgorithmsfor

convexoptimization”,ComputerVisionandImageUnderstanding(2011).

doi:10.1016/j.cviu.2011.06.011.

[55] P.L.Combettes,J.-C.Pesquet,Aproximaldecompositionmethodforsolving

convexvariationalinverseproblems,InverseProblems24(2008)1–27.

[56] P.L.Combettes,Iterativeconstructionoftheresolventofasumofmaximal

monotoneoperators,J.ConvexAnal.16(2009)727–748.

69

[57] J.Yang,Y.Zhang,W.Yin,AfastalternatingdirectionmethodforTVL1-L2

signalreconstructionfrompartialfourierdata,IEEEJ.Sel.Top.Signal

Process.4(2)(2010)288–297.

[58] J.Liu,P.Musialski,P.Wonka,J.Ye,Tensorcompletionforestimating

missingvaluesinvisualdata,in:ProceedingsofICCV,2009.

[59] S.Wright,R.Nowak,M.Figueiredo,Sparsereconstructionbyseparable

approximation,IEEETrans.SignalProcess.57(7)(2009)2479–2493.

[60] J.Eckstein,B.F.Svaiter,Generalprojectivesplittingmethodsforsumsof

maximalmonotoneoperators,SIAMJ.ControlOptim.48(2009)787–811.

[61] D.Gabay,B.Mercier,Adualalgorithmforthesolutionofnonlinear

variationalproblemsviafinite-elementapproximations,Comput.Math.

Appl.2(1976)17–40.

[62] R.Glowinski,P.L.Tallec,AugmentedLagrangianandoperator-splitting

methodsinnonlinearmechanics,SIAMStud.Appl.Math.9(1989).

[63] P.Tseng,Applicationsofasplittingalgorithmtodecompositioninconvex

programmingandvariationalinequalities,SIAMJ.ControlOptim.29

(1991)119–138.

[64] B.S.He,L.-Z.Liao,D.Han,H.Yang,Anewinexactalternatingdirection

methodformonotonevariationalinequalities,Math.Program.92(2002)

103–118.

[65] S.Vasanawala,M.Alley,R.Barth,B.Hargreaves,J.Pauly,andM.Lustig.

FasterpediatricMRIviacompressedsensing.InProc.AnnualMeetingSoc.

PediatricRadiology(SPR),Carlsbad,CA,Apr.2009.

[66] M.YuanandY.Lin.Modelselectionandestimationinregressionwith

groupedvariables.J.Roy.Stat.Soc.Ser.B,68(1):49{67,2006.

[67] M.Davenport.Randomobservationsonrandomobservations:Sparse

signalacquisitionandprocessing.PhDthesis,RiceUniversity,Aug.2010.

[68] Y.Chi,L.Scharf,A.Pezeshki,andR.Calderbank.Sensitivitytobasis

mismatchincompressedsensing.Preprint,May2010.

70

[69] A.Cohen,W.Dahmen,andR.DeVore.Instanceoptimaldecodingby

thresholdingincompressedsensing.InInt.Conf.HarmonicAnalysisand

PartialDifferentialEquations,Madrid,Spain,Jun.2008.

[70] A.Cohen,W.Dahmen,andR.DeVore.Compressedsensingandbestk-term

approximation.J.Amer.Math.Soc.,22(1):211{231,2009.

[71] S.Mun,J.E.Fowler,“Blockcompressedsensingofimagesusingdirectional

transforms.”,IEEEInternationalConferenceonImageProcessing(ICIP),

IEEE,pp.3021–3024.(2009)

[72] E.Candes,J.Romberg,“Sparsityandincoherenceincompressivesampling.

Inverseproblems”,23(3):969.(2007)

[73] N.Ahmed,T.Natarajan,K.R.Rao,“Discretecosinetransform.”,IEEE

TransactionsonComputers100(1):90–93.(1974)

[74] J.Trzasko,A.Manduca,“AUnifiedTensorRegressionFrameworkfor

CalibrationlessDynamic,Multi-ChannelMRIReconstruction”;2013;Salt

LakeCity,USA.(2013).

[75] C.Prieto,M.Usman,J.M.Wild,S.Kozerke,P.G.Batchelor,“Groupsparse

reconstructionusingintensity-basedclustering.”MagneticResonancein

Medicine69:1169–1179.(2012)

[76] WangS-J,SunM-F,ChenY-H,PangE-P,ZhouC-G(2012)STPCA:sparse

tensorprincipalcomponentanalysisforfeatureextraction.Pattern

Recognition(ICPR),201221stInternationalConferenceon.TsukubaJapan.

2278–2281.

[77] P.Comon,“Tensordecompositions.”,MathematicsinSignalProcessingV:

1–24.

[78] D.Liang,EVR.DiBella,R.RChen,L.Ying“K-TISD:Compressedsensingwith

iterativesupportdetectionfordynamicMRI.BiomedicalImaging:From

NanotoMacro,2011IEEEInternationalSymposiumon:IEEE.1264–1267.

(2011).

[79] M.Akcakaya,T.A.Basha,B.Goddu,L.A.Goepfert,K.V.Kissinger,“Low-

dimensional-structureself-learningandthresholding:Regularization

71

beyondcompressedsensingforMRIReconstruction.”,MagneticResonance

inMedicine66:756–767.(2011)

[80] T.G.Kolda,B.W.Bader,“Tensordecompositionsandapplications.”,SIAM

review51:455–500.(2009)

[81] F.L.Hitchcock,“Multipleinvariantsandgeneralizedrankofap-waymatrix

ortensor.”JournalofMathematicsandPhysics7:39–79.(1927)

[82] WangS-J,YangJ,SunM-F,PengX-J,SunM-M,etal.(2012)Sparsetensor

discriminantcolorspaceforfaceverification.NeuralNetworksand

LearningSystems,IEEETransactionson23:876–888.

[83] W.F.Sacco,H.A.Filho,P.Claudio,“Cost-basedoptimizationofnuclear

reactorcoredesign:Apreliminarymodel”,2007

[84] A.K.Cline,I.S.Dhillon,“ComputationoftheSingularValueDecomposition”,

bookchapter45inHandbookofLinearAlgebra,TheUniversityofTexasat

Austin.

72

Appendix

ListofAlgorithms

Aspartofthisthesis,someofthewell-knownalgorithmsareusedandsomealso

experimentedwith.Herearethelistofalgorithmsnamesandtheirpseudocode.

Allthesepseudocodeisreferencefrom[54].

1) IterativeShrinkage-ThresholdingAlgorithm(ISTA)[26]

Algorithm1.ISTA

Input:𝜌 = 1 𝐿! , 𝑥!

repeat

fork=1toKdo

𝑥! = 𝑝𝑟𝑜𝑥!(𝑔)(𝑥!!! − 𝜌∇𝑓(𝑥!!!))

endfor

untilStopcriterions

2) FastIterativeShrinkage-ThresholdingAlgorithm(FISTA)[26]

Algorithm2.FISTA

Input:𝜌 = 1 𝐿! ,𝑟! = 𝑥!,𝑡! = 1

repeat

fork=1toKdo

𝑥! = 𝑝𝑟𝑜𝑥!(𝑔)(𝑟! − 𝜌∇𝑓(𝑟!))

𝑡!!! = !! !!!(!!)!

!

𝑟!!! = 𝑥! + !!!!!!!!

(𝑥! − 𝑥!!!)

73

endfor

untilStopcriterions

3) CompositeSplittingDenoising(CSD)[55,56]

Algorithm3.CSD

Input:𝜌 = 1 𝐿,𝛼,𝛽,{𝑧!!}!!!,….,! = 𝑥!

repeat

forj=1toJdo

fori=1tomdo

𝑥! = 𝑎𝑟𝑔𝑚𝑖𝑛! !!!"

||𝑥 − 𝑧!!!!||! + 𝑔!(𝐵!𝑥)

endfor

𝑥! = !!

𝑥!!!!!

fori=1tomdo

𝑧!! = 𝑧!

!!! + 𝑥! − 𝑥!

endfor

endfor

untilStopcriterions

4) CompositeSplittingAlgorithm(CSA)[55,56]

Algorithm4.CSA

Input:𝜌 = 1 𝐿,𝑥!

repeat

fork=1toKdo

fori=1tomdo

𝑦!! = 𝑝𝑟𝑜𝑥! 𝑔! (𝐵! (𝑥!!! − !!∇𝑓! 𝑥!!! ))

endfor

74

𝑥! = !!

𝐵!!!!!!! 𝑦!!

endfor

untilStopcriterions

5) Fastcompositesplittingalgorithms(FCSA)[26]

Algorithm5.FCSA

Input:𝜌 = 1 𝐿,𝑟! = 𝑥!,𝑡! = 1

repeat

fork=1toKdo

fori=1tomdo

𝑦!! = 𝑝𝑟𝑜𝑥! 𝑔! (𝐵! (𝑟! − !!∇𝑓! 𝑟! ))

endfor

𝑥! = !!

𝐵!!!!!!! 𝑦!!

𝑡!!! = !! !!!(!!)!

!

𝑟!!! = 𝑥! + !!!!!!!!

(𝑥! − 𝑥!!!)

endfor

untilStopcriterions

6) FastcompositesplittingalgorithmsusedforCompressedSensing(FCSA-

MRI)[20]

Algorithm6.FCSA-MRI

Input:𝜌 = 1 𝐿 ,𝛼,𝛽,𝑟! = 𝑥!,𝑡! = 1

repeat

75

fork=1toKdo

𝑥! = 𝑟! − 𝜌∇𝑓 𝑟!

𝑥! = 𝑝𝑟𝑜𝑥!(2𝛼||𝑥||!")(𝑥!)

𝑥! = 𝑝𝑟𝑜𝑥!(2𝛽||Φ𝑥||!)(𝑥!)

𝑥! = (𝑥! + 𝑥!) 2

𝑡!!! = !! !!!(!!)!

!

𝑟!!! = 𝑥! + !!!!!!!!

(𝑥! − 𝑥!!!)

endfor

untilStopcriterions

7) FastcompositesplittingalgorithmswithLowRankTensorCompletion

(FCSA-LRTC)[58]

Algorithm7.FCSA-LRTC

Input:𝜌 = 1 𝐿! ,𝛼,𝛽,𝑟! = 𝑥!,𝑡! = 1

repeat

fork=1toKdo

fori=1tomdo

𝑦!! = 𝑝𝑟𝑜𝑥! 𝛼!||𝑥!||∗ (𝑟!! − 𝜌𝐴∗(𝐴(𝑟!! − 𝑏)))

endfor

𝑥! = !!

𝐵!!!!!! 𝑦!!

𝑡!!! = !! !!!(!!)!

!

𝑟!!! = 𝑥! + !!!!!!!!

(𝑥! − 𝑥!!!)

endfor

untilStopcriterions

76

8) SingularValueDecomposition(SVD)[43,84]

Algorithm8.BiorthogonalizationSVD

Input:𝑚,𝑛,𝐴where𝐴is𝑚 ×𝑛.

Output:Σ,U,𝑉sothatΣisdiagonal,Uand𝑉haveorthonormalcolumns,U

is𝑚 ×𝑛,𝑉is𝑛 ×𝑛,and𝐴=U Σ𝑉! .

1. U ← 𝐴.(Thisstepcanbeomittedif𝐴istobeoverwrittenwithU)

2. 𝑉 = 𝐼! × !

3. Set𝑁! = ( 𝑢!,!!!!!!

!!!! ),𝑠 = 0,and𝑓𝑖𝑟𝑠𝑡 = 𝒕𝒓𝒖𝒆

4. Repeatuntil𝑠! ! ≤ 𝜀!𝑁!and𝑓𝑖𝑟𝑠𝑡 = 𝒇𝒂𝒍𝒔𝒆.

a. Set𝑠 = 0and𝑓𝑖𝑟𝑠𝑡 = 𝒇𝒂𝒍𝒔𝒆.

b. for𝑖 = 1,… . .𝑛 − 1

i. for𝑗 = 𝑖 + 1,… . ., 𝑛

• 𝑠 ← 𝑠 + ( 𝑢!,!𝑢!,!!!!! )!

• Determine𝑑!,𝑑!, 𝑐 = cos (𝜃),and𝑠 = sin (𝜑)such

that:

𝑐 −𝑠𝑠 𝑐

𝑢!,!!!

!!!

𝑢!,!𝑢!.!

!

!!!

𝑢!,!𝑢!.!

!

!!!

𝑢!,!!!

!!!

𝑐 𝑠−𝑠 𝑐 = 𝑑! 0

0 𝑑!

• U ← 𝑈𝑅!,! 𝑐, 𝑠 where𝑅!,! 𝑐, 𝑠 istheGivens

rotationmatrixthatactsoncolumns𝑖and𝑗during

rightmultiplication.

• 𝑉 ← 𝑉𝑅!,! 𝑐, 𝑠

5. for𝑖 = 1,… . .𝑛

a. 𝜎! = 𝑢!,!!!!!!

b. 𝑈 ← 𝑈Σ!!

77

9) PrincipalComponentAnalysis(PCA)[39,83]

Algorithm9.PCA

Generateaninitialsolution𝑂𝑙𝑑_𝐶𝑜𝑛𝑓𝑖𝑔

for𝑛 = 0to⋕ofiterations

Generateastochasticperturbationofthesolution

𝑖𝑓Fitness(𝑁𝑒𝑤_𝐶𝑜𝑛𝑓𝑖𝑔)>Fitness(𝑂𝑙𝑑_𝐶𝑜𝑛𝑓𝑖𝑔)

𝑂𝑙𝑑_𝐶𝑜𝑛𝑓𝑖𝑔:=𝑁𝑒𝑤_𝐶𝑜𝑛𝑓𝑖𝑔

Exploration()

𝑒𝑙𝑠𝑒

Scattering()

𝑒𝑛𝑑 𝑖𝑓

𝑒𝑛𝑑for

Exploration()

for𝑛 = 0to⋕ofiterations

Generateasmallstochasticperturbationofthesolution

𝑖𝑓Fitness(𝑁𝑒𝑤_𝐶𝑜𝑛𝑓𝑖𝑔)>Fitness(𝑂𝑙𝑑_𝐶𝑜𝑛𝑓𝑖𝑔)

𝑂𝑙𝑑_𝐶𝑜𝑛𝑓𝑖𝑔:=𝑁𝑒𝑤_𝐶𝑜𝑛𝑓𝑖𝑔

𝑒𝑛𝑑 𝑖𝑓

𝑒𝑛𝑑for

return

Scattering()

𝑝!"#$$%&'() = 1− 𝐹𝑖𝑡𝑛𝑒𝑠𝑠(𝑁𝑒𝑤_𝐶𝑜𝑛𝑓𝑖𝑔)

𝐵𝑒𝑠𝑡 𝐹𝑖𝑡𝑛𝑒𝑠𝑠

𝑖𝑓𝑝!"#$$%&'() > 𝑟𝑎𝑛𝑑𝑜𝑚 0, 1

𝑂𝑙𝑑_𝐶𝑜𝑛𝑓𝑖𝑔: = 𝑟𝑎𝑛𝑑𝑜𝑚 𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛

78

𝑒𝑙𝑠𝑒

Exploration()

𝑒𝑛𝑑 𝑖𝑓return

Documents

NEW ALGORITHMS FOR COMPRESSED SENSING OF MRI: …