1

HighlyAcceleratedMultishotEPIthrough

SynergisticMachineLearningandJointReconstruction

BerkinBilgic1,2,3,ItthiChatnuntawech4,MaryKateManhard1,2,QiyuanTian1,2,CongyuLiao1,2,StephenF.

Cauley1,2,SusieY.Huang1,2,3,JonathanR.Polimeni1,2,3,LawrenceL.Wald1,2,3,KawinSetsompop1,2,3

1AthinoulaA.MartinosCenterforBiomedicalImaging,Charlestown,MA,USA

2DepartmentofRadiology,HarvardMedicalSchool,Boston,MA,USA

3Harvard-MITHealthSciencesandTechnology,MIT,Cambridge,MA,USA

4NationalNanotechnologyCenter,NationalScienceandTechnologyDevelopmentAgency,PathumThani,

Thailand

Correspondingauthor:

ItthiChatnuntawech

itthi.cha@nanotec.or.th

Bodywordcount:~6400

Keywords:

MultishotEPI,parallelimaging,machinelearning,deeplearning,convolutionalneuralnetwork,joint

reconstruction

2

ABSTRACT

Purpose: To introduce a combined machine learning (ML) and physics-based image reconstruction

framework that enables navigator-free, highly acceleratedmultishot echo planar imaging (msEPI), and

demonstrateitsapplicationinhigh-resolutionstructuralanddiffusionimaging.

Methods:SingleshotEPIisanefficientencodingtechnique,butdoesnotlenditselfwelltohigh-resolution

imaging due to severe distortion artifacts and blurring.WhilemsEPI canmitigate these artifacts, high-

quality msEPI has been elusive because of phase mismatch arising from shot-to-shot variations which

preclude the combination of the multiple-shot data into a single image.We employ deep learning to

obtainaninterimimagewithminimalartifacts,whichpermitsestimationofimagephasevariationsdueto

shot-to-shotchanges.Thesevariationsarethen includedinaJointVirtualCoilSensitivityEncoding(JVC-

SENSE)reconstructiontoutilizedatafromallshotsandimproveupontheMLsolution.

Results:Our combinedML+physicsapproachenabledRinplane xMultiBand (MB)=8x2-foldacceleration

using2EPI-shotsformulti-echoimaging,sothatwhole-brainT2andT2*parametermapscouldbederived

from an 8.3 sec acquisition at 1x1x3mm3 resolution. This has also allowed high-resolution diffusion

imagingwithhighgeometricfidelityusing5-shotsatRinplanexMB=9x2-foldacceleration.Tomakethese

possible,weextendedthestate-of-the-artMUSSELSreconstructiontechniquetoSimultaneousMultiSlice

(SMS)encodinganduseditasaninputtoourMLnetwork.

Conclusion:CombinationofMLandJVC-SENSEenablednavigator-freemsEPIathigheraccelerationsthan

previouslypossiblewhileusing fewershots,withreducedvulnerability topoorgeneralizabilityandpoor

acceptanceofend-to-endMLapproaches.

3

INTRODUCTION

Slow imageencodinghasconstrainedclinicalMRI scans touse2-dimensionalencodingand thick slices,

often with slice gaps, so that whole-brain exams can be completed within acceptable time frames. In

addition to the information loss, such inefficient acquisition poses a barrier to MRI evaluation of

hospitalizedpatientswhoare critically ill andcanneitherhold still nor tolerate long scans.Lowpatient

throughputduetoinefficientimagingalsoincreasesthetimefromsymptomonsettodiagnosis,thereby

delayingtreatment.

To overcome the slow image encoding barrier, recent screening protocols havemoved to singleshot

Echo Planar Imaging (ssEPI) to provide multi-contrast information (1,2). Unfortunately, the reduced

geometric fidelityof theseprotocolsmayconfound/obscure localizationof salient imaging findings.The

problemarisesfromseveredistortionandblurringartifactsinssEPIathighin-planeresolutions,wherea

largeareaofk-spacehastobecoveredwithinasinglereadoutinthepresenceofB0inhomogeneityand

T2* signal decay. These effects are only partiallymitigated at relatively high in-plane acceleration (e.g.

Rinplane=3).

Whilemultishot EPI (msEPI) canmitigateblurring anddistortion, high-qualitymsEPI has beenelusive

becausecombiningthemultiple-shotdataintoasingleimageisprohibitivelydifficult,especiallyathighin-

plane acceleration. Image phase mismatches between the shots caused by physiological variations

(respiration, cardiac pulsation) or motion under the influence of diffusion encoding gradients lead tosevereghostingartifacts.Todate,theapplicationofmsEPIhasbeenrestrictedtodiffusionimaging,where

two types of solutions have been proposed to combine the shots: (i) navigator-based approaches that

requireadditionaldataacquisitiontocaptureshot-to-shotphasevariations (3–7),and(ii)navigator-free

techniques that estimate these variations from the data itself (8–11). In (ii), multiplexed sensitivity

encoding (MUSE) (9) and its extensions (12,13), relyonparallel imaging to reconstruct an intermediate

imageforeachshotindependentlytoestimatethephysiologicalvariationsbeforejointlyreconstructingall

multishotdatatogether.This limitstheachievabledistortionandblurringreductionto4to6-fold,since

parallel imagingwithmodernRFreceivecoilarraysbreaksdownbeyondsuchaccelerationinthephase-

encoding direction.MUSSELS, on the other hand, does not explicitly estimate the phase of each shot

image,butemploys sensitivityencodingandsimilaritiesacrossmultishotdata in the formof structured

low-rankmatrixcompletion(11,14).ThishasallowedMUSSELStoundersamplethek-spaceofeachshot

byRinplane=8-foldtoreducedistortionandblurringartifactsaswellastheechotime(TE).Imagescouldbe

successfullyreconstructedusing4-shotsofdata,sothatthenetaccelerationfactorbecameRnet=8/4=2-

fold.Itisimportanttonoteanotherclassofnavigator-freemultishotdiffusionimagingtechniques,which

utilize non-Cartesian trajectories that allow for estimation of low-resolution image phase information

from thedensely sampledportionofeach shot (15,16). Such self-navigationpropertymaycomeat the

costofblurring/distortionintheresultingimages.

4

In this contribution, we introduce a new reconstruction framework that utilizes a synergistic

combination of machine learning (ML) and physics (or forward-model) based reconstruction, and

demonstrate itsapplicationinstructuralanddiffusionmsEPIwithhighgeometricfidelity.Wetermour

combinedML + physics approachNetwork EstimatedArtifacts for Tempered Reconstruction (NEATR),

and incorporate Simultaneous MultiSlice (SMS) for extra efficiency. To this end, we have extended

MUSSELStoSMSencoding,andutilizedthereadoutextendedFOVconcept(17)toseamlesslyintegrate

sliceaccelerationintothisframework.WestartfromSMS-MUSSELSreconstructionofhighlyaccelerated

msEPIusingasmallernumberofacquisitionshots,andpasstheintermediatesolutionthroughourdeep

neural network tomitigate the reconstruction artifacts from SMS-MUSSELS. Using this interim image

with minimal artifacts allows us to solve for the image phase of each shot using phase-regularized

parallel imaging (18). Given the phase of each shot, we then perform a Joint Virtual Coil Sensitivity

Encoding(JVC-SENSE)reconstructionwhereweutilizethek-spacedatafromallshotsaswellasvirtual

coilconcept(19–21)tosolveforthecombinedmagnitudeimage.

We demonstrate the application of SMS-NEATR in spin-and-gradient-echo (SAGE (22)) msEPI

acquisition at Rinplane xMultiBand (MB) = 8x2-fold acceleration using 2-shots. Compared to the newly

developedSMS-MUSSELSreconstruction,whichisalsousedasaninputtoournetwork,wedemonstrate

~30%improvementinroot-mean-squarederror(RMSE)inhigh-resolutionstructuralimages.Weobserve

larger gains in ghosting/aliasing artifactmitigation in the harder problem of diffusion imaging, where

SMS-NEATRallowsforRinplanexMB=9x2-foldaccelerationusing5-shots.

These are made possible by the deep learning step that enables phase estimations at such high

accelerationfactors.Importantly,thefinaluseofarigorousphysics-basedforward-modelreconstruction

limits theroleofML in the final reconstruction.Thus,SMS-NEATRallowsus to tap intothepotentialof

convolutionalneuralnetworks (CNN)tosolve for importantnuisancemodulationsandunknowns in the

forward model without treating the reconstruction as an end-to-end process. The result is a better

harnessedsensitivityencodingwithfullutilizationofthescannerhardware.Ourstrategypavesthewayto

reaping the benefits of ML while constraining potential damage from utilizing it on data beyond its

training experience, andwithout being exposed to the vulnerabilities of not knowing exactly what the

reconstruction is doing. Our approach of using ML to estimate nuisance parameters that are hard to

determine could allow physics-based reconstructions to work well in other applications, such as

retrospectivemotion-correctionwithoutnavigationoradditionalhardware.

We provide Matlab source code and data to reproduce our diffusion msEPI results here:

https://bit.ly/2QgBg9U

SupportingInformationfigurescanbeaccessedhere:https://bit.ly/2unY2iJ

5

METHODS

ReconstructionOverview

TheSMS-NEATRflowchartispresentedinFig.1.WebeginbyperforminganSMS-MUSSELSreconstruction

on the highly-accelerated (e.g. Rinplane xMB = 8x2)msEPI data to obtain an initial image estimatewith

mitigatedartifacts from thenuisancephasebetween shots. The image is further improvedusingU-Net

processing(23)whichestimatesarefinedimagewithminimalartifacts.Startingfromthisreconstruction,

weestimate thephase imagecorresponding toeachshotusingphase-regularizedparallel imaging (24).

Giventheestimatedshot-to-shotphasevariations,wethenperformaphysics-basedjointreconstruction

(JVC-SENSE)toarriveatthefinalsolution.JVC-SENSEincorporatessliceaccelerationandusesk-spacedata

fromallshotsandtheirconjugatesymmetriccounterpartstosolveforacommonmagnitudeimage.We

detailtheindividualstepsnext.

SMS-MUSSELSFormalism

ThefirststepofSMS-NEATRisbasedonaMUSSELSreconstruction,wheretheinputistheacquiredmulti-

shot k-space data, and the output is an estimate of shot imageswhich are further refined in the later

steps.Tobeginwith,wewill ignoreSMSencodingandconsideronly in-planeacceleration. In this case,

MUSSELSentailsthesolutionofthefollowingoptimizationproblem:

𝑚𝑖𝑛𝒙 𝐹&𝐶𝑥& − 𝑑& ++

,-

&./+ 𝜆 ℋ(𝒙) ∗

Eq1

where𝐹& represents theundersampleddiscreteFourier transform(DFT)corresponding toshot𝑡,𝐶 arethecoilsensitivities,𝑥& istheunknowncomplex-valuedimageinshot𝑡withsize𝑁/×𝑁+,and𝑑&aretheacquiredk-spacedatainthisshot.Theterm 𝐹&𝐶𝑥& − 𝑑& +

+thusrepresentsourdataconsistencythrough

sensitivityencoding(25).Theoperatorℋ(∙)firstappliestheDFT,andthenextracts𝑟×𝑟×𝑁;patchesink-spacetogenerateadatamatrixℋ(𝒙)withblock-wiseHankelstructure(11,26–28).Thisoperatoractsona 3-dimensional data structure𝒙 of size𝑁/×𝑁+×𝑁;, which is formed by concatenating the images𝑥&fromall𝑁;shotstogether.Thenuclearnormconstraint ℋ(𝒙) ∗thusenforcesa low-rankprioronthe

Figure1 SMS-NEATR is a combinedmachine learningandphysics-based reconstruction technique for highly-acceleratedmsEPIacquisition.We developed SMS-MUSSELS algorithm to provide an initial solution, whichmay suffer from artifacts due to highacceleration(RinplanexMB=8x2with2-shots).Startingfromthis,residuallearningwithU-Netarchitectureprovidesaninterimimagewithminimalartifacts.Giventhissolution,phasecyclingalgorithm isusedforestimatingshot-phases,whicharethenutilizedassensitivityvariationsinafinaljointvirtualcoil(JVC)SENSEreconstruction.

6

block-Hankelrepresentationofthemultishotdataink-space.ThispriorissimilartotheSAKEformulation

(28), albeit with two differences: the coil axis is now replaced by the shot dimension, and sensitivity

encodingisexplicitlyexploited.Assuch,wefollowtheSAKEapproachandpursueasimple,POCS-SENSE

likealgorithm(29)tosolveEq1asdetailedintheAppendix.WewillshowthattheadvancedFISTAupdate

rules(30) improveconvergenceandimagequality.Wealsonotethatthecostfunctionbeingminimized

differsfromtheconvexoptimizationproblemsetbytheoriginalMUSSELSapproach,andismoresimilar

toLORAKS-typeapproaches(31)astheysolveanon-convexproblem.

ExtensiontoSMS:WedevelopedanewapproachtoallowMUSSELStoworkwithSMSencodingusingthe

readout-extendedFOVconcept(17).ThisrepresentsSMSasundersamplinginthekxaxisbyconcatenating

thetwoslicesalongthereadout(Fig.1a).In-planeandsliceaccelerationcouldthusbecapturedusingthe

Fourieroperator𝐹&inEq1,nowwithsimultaneouskx-kyundersampling.

InthenextstepoftheSMS-NEATRreconstruction,weusetheestimatedshotimages 𝑥& &./,- asinput

to a residual CNN (32,33) with U-Net architecture (23). The network aims to learn and mitigate the

reconstructionerrorsinSMS-MUSSELSandprovideoutputshotimages, 𝑢& &./,- ,withminimalartifacts.

NetworkArchitecture

Weusedapatch-basedU-Nettolearnthemappingbetweentheinitialreconstructionanditsdifference

tothegroundtruth image inaslice-by-slicemanner.Thenetworkconsistedof5 levels (Fig.2),andthe

numberofconvolutionalfilterswas64atthehighestlevel.Asthesizeofinputwasreduced2-foldbymax

poolinginthenext level,thenumberoffilterswasincreased2-foldtoretainthetotalnumberofkernel

weightsineachlevel.Thekernelshadsize3x3,andeachdropoutlayersetarandomlyselected5%ofits

input units to zero to help avoid overfitting (34). Leaky ReLUwas selected as the nonlinear activation

function (35). Batch normalization (BN) was utilized to help accelerate training and avoid saturating

nonlinearities(36).

Figure2U-Netarchitectureisusedtolearnthemappingbetweenpatchesofshot-imagesreconstructedwithSMS-MUSSELS,andtheir difference to reference data. Both the input and output have been decomposed into real and imaginary components toenablecomplex-valuedprocessing forSAGEreconstruction.64×64patchesfromall theshotsarepresentedasinputtoa5-levelnetwork,wherethefirstleveluses64convolutionalfilters.Tohelpprovidescaleinvariance,maxpoolingoperatorsdownsamplethepatchesaftereachlayer.Atthesametime,thenumberoffiltersaredoubledtoretainthetotalnumberofkernelweightsateachlevel.

7

ForSAGEreconstruction,wetrainedacomplex-valuednetworkbyseparatingeachofthe2-shotimages

into its real and imaginary components. Thiswayweused4 channels, ℜ 𝑥/ , ℑ 𝑥/ , ℜ 𝑥+ , ℑ(𝑥+) , in

this network configuration (Fig. 2). For DWI, we explored using both complex-valued and magnitude-

basednetworks.Thecomplexnetworkmadeuseof10channels(realandimaginarycomponentsfrom5-

shots)whiletheabsolutevalueofeachofthe5-shots, 𝑥/ , … , 𝑥A wasusedasinputchannelsforthe

magnitude-basedmodel.Theremainderofthemanuscriptfocusesonthemagnitude-basedDWInetwork,

andthecomplex-valuedU-NetresultsarereportedinSupportingInformationFigS9andS10.

NetworkImplementation

Kerasprogramming interface (37)withTensorflow (38)backendwerechosen toperform the training.

ADAMoptimizer(39)wasusedwithlearningrate=0.001anddecay=0.001.Whilelearningrateactsas

agradientdescentstepsize,decayparameterdampensthisstepsizetotakeprogressivelysmallersteps

ineachepoch.Anℓ+lossfunctionwasminimizedusing200epochsandabatchsizeof128.AnNVIDIA

TitanXPgraphicscardwith12GBmemorywasusedfortraining,whichtook~18hoursforSAGEand~19

hoursforDWI.

ForSAGEprocessing,slicesandechoesweretreatedasdifferenttraininginstances.Attheteststage,

thepatch-basednetworkwasappliedinasliding-windowmannerwithastepsizeof10voxels,andthe

estimated residuals from overlapping patches were averaged together. This process took 4.5

seconds/slice.SimilarlyforDWI,slicesanddiffusiondirectionsweretreatedasdifferenttrainingsamples

andtheinferencetook9seconds/slice.

Phasecycling

WeformamagnitudeestimatefromtheU-Netshotimagesbyaveraging,𝑚CDEF =/,-

𝑢&,-&./ .Wekeep

this improved magnitude fixed, and solve only for the image phase of each EPI-shot𝜙& using phase-regularizedparallelimaging,orphasecycling(24):

𝑚𝑖𝑛IJ 𝐹&𝐶𝑚CDEF ∙ 𝑒LIJ − 𝑑& +++ 𝛼 𝑊𝜙& / Eq2

Here,only thehighlightedphase information𝜙& isunknown,𝑊 is awaveletoperator that imposes

sparsity prior on the shot phase via ℓ/ penalty, and 𝛼 is a parameter that controls the degree of

regularization. The solution of this problem is made easier by the fact that we are using sensitivity

encoding to solve only for the real-valued individual shot phases rather than the complex-valued

individual shot images. The shot phases from the complex-valued SAGE network,∢𝑢&, were used toinitializethisnon-convexproblem.ForDWIreconstructionwithmagnitude-basedU-Netprocessing,shot

phases from SMS-MUSSELS reconstruction, ∢𝑥&, served as initial guess. The complex-valued DWI

network was still able to provide shot phase information,∢𝑢&, to initialize phase cycling (SupportingInformation Figs S9 and S10). SMS acceleration is again embedded in Eq2 via the 2-dimensional

undersampling in𝐹&, and the coil sensitivities of the slices concatenated in the readout direction as

8

representedby𝐶.

JVC-SENSE

Givenestimatesofshot-to-shotphasevariations𝜙&,wecannowjointlysolveforthecommonmagnitude

image𝑚 using the data from all shots, through harnessing sensitivity encoding for slice and in-plane

acceleration(25)andthevirtualcoil(VC)concept(19,20)inJVC-SENSE.Todothis,wesolveasimpleleast

squaresproblem:

𝑚𝑖𝑛P𝐹&𝐶𝑒LIJ

𝐹Q&𝐶∗𝑒QLIJ𝑚 −

𝑑&𝑑Q&∗ +

+,-

&./+ 𝛽 ∙ ℛ(𝑚)

Eq3

Here,theonlyunknownisthehighlightedmagnitudeimage𝑚,andthecoilsensitivitiesaremodified

toincludethephasevariationineachshottoyieldthecombinedsensitivities𝐶𝑒LIJ .TheVCconceptisenforced by augmenting the optimization with the conjugate symmetric k-space data 𝑑Q&∗ and the

conjugatesensitivities𝐶∗𝑒QLIJ .Conjugatesymmetrick-spaceisderivedfromtheacquiredk-spacedata

bycomplex-conjugationandflippingtheaxesinthekx-kyplane.Wehaveusedtheshorthandnotation– 𝑡toexpressthismirroringoperationink-space.Jointreconstructionacrossallshotsisperformedviathe

summationoperator (∙),-&./ .ForstructuralimagingwithSAGE,wehaveusedtotalvariation(TV)penalty

as the regularizerℛ(∙),with the corresponding regularization parameter𝛽. For diffusion imaging,we

haveexploredusingTVregularizationaswellasasimpleTikhonovpenalty.

DuringthereviewofthispaperandafterpublicdisseminationofSMS-NEATRpreprint(40),abstract(41)andcode(https://bit.ly/2QgBg9U)whichintroducedSMS-MUSSELSanditsimprovementusingdeeplearning,twopreprintshavealsoappearedthatdescribeSMS-MUSSELSandanalternativedeeplearningenhancedmsEPIreconstruction(42,43).

TrainingData

Spin-and-gradient-echo(SAGE)

In compliancewith Institutional ReviewBoard (IRB) requirements, three volunteerswere scannedon a

SiemensPrisma3TsystemwithSAGE(22)msEPIsequencetobuildatrainingdataset.Multishotdatawere

collected, where each shot was acquired at Rinplane=8-fold acceleration, and a total of 8-shots were

collected with a ∆ky sampling shift between the shots. When combined, this corresponded to a fully-

encoded acquisition at Rnet=1. Relevant parameters were: field of view (FOV) = 220x220x120 mm3,

resolution = 1x1x3mm3, echo times (TEs) = 26/61/95/130/165ms, repetition time (TR) = 8.3 sec, and

effectiveechospacing=0.148ms.Eachshotsampledonly27phaseencodinglinesduetoRinplane=8-fold

acceleration. First twoechoeswere sampledbefore the180˚pulse, and the latter threewereacquired

aftertherefocusingpulse.Thefifthechowastimedsothatitwasaspinechoimage.

9

Coil sensitivities used in reconstructions were estimated using ESPIRiT (44,45) based on a FLEET

acquisition (46). FLEETautocalibration signal (ACS) acquisition collectsmultishot gradient echoEPIdata

withlowflipangles,wherealltheshotsforaspecificsliceareacquiredfirst.Thenallshotsforthesecond

slice are sampled, and this is repeateduntil every slice in the FOVprescription are accounted for. This

way,encodingofeachsliceiscompletedwithinatimeframeontheorderof100msec,andshot-to-shot

variations are minimized. Unlike the FLEET calibration scan, the “standard mode” for ACS acquisition

wouldsamplethe1stshotforalltheslicesfirst,thenacquirethe2ndshotagainforallslicepositions.The

time frame for samplingall the shots is thuson theorder seconds,which increases thevulnerability to

shot-to-shotmotionandhasdetrimentalimpactontheACSdataquality(46).UsingFLEETacquisitionhas

thusallowedus to improve the robustnessofourcoil sensitivityestimation.Allacquisitionsweremade

withaSiemens32-channelheadcoil.

To obtain clean reference data, MUSSELS reconstruction was performed using all of the 8-shots at

Rinplane=8 acceleration,which yielded “fully-sampled” ground-truth images. To enable higher acquisition

efficiency,only2-shotsoutofthe8-shotdatawereselectedforsubsampledreconstructions.The2-shots

were acquiredwith a k-space shift of ∆ky=4 samples to provide complementary coverage. Thesewere

furthercollapsedintheslicedirectiontosimulateMB=2-foldacceleration,sothatthetotalacceleration

factorpershotbecameRinplanexMB=8x2.ThishighlyundersampledmsEPIdatawerethenreconstructed

using SMS-MUSSELS. Due to the very high acceleration rates, SMS-MUSSELS algorithm incurred

reconstructionartifacts.Theseerrorswithrespecttothecleanreferenceimagewereusedasthetraining

targetinourresiduallearningapproach(Fig.2).

Weextracted57600overlappingpatchesofsize64×64withastepsizeof16voxelsfromthetraining

data.The2-shotsdecomposedintorealandimaginarycomponentsintheSAGEacquisitionweretreated

as input channels, andwere concatenated to create64×64×4patches thatwere fed to thenetwork to

enable joint reconstruction across shots. The training dataset was enriched by 16-fold using

augmentations including scaling (0.5×, 1×, 2×), flipping the axes (left-right, anterior-posterior and echo

dimension)androtations(±135,±90,±45degrees).

DiffusionWeightedImaging(DWI)

Threevolunteerswerescannedona3TPrismasystemtobuildupaDWItrainingdataset,consistingof9-

shotdataacquiredatRinplane=9-foldacceleration.Theparameterswere:fieldofview(FOV)=224x224x120

mm3, resolution = 1x1x3mm3, TE/TR = 54/5100 ms, and effective echo spacing = 0.13 ms. Each shot

sampledonly24phaseencoding linesdue toRinplane=9-foldacceleration. Inaddition toab=0 image, six

diffusiondirectionsatb=1000s/mm2werecollected.FLEETcalibrationdatawereusedtoestimateESPIRiT

coilsensitivitymaps.

Reference “fully-sampled” images were obtained using all 9-shots in MUSSELS reconstruction.

Subsampledacquisitionswereobtainedbyselecting5-shotsoutofthis9-shotdataset.The5-shotswere

shifted by ∆ky={0,2,4,6,8} samples to provide complementary information. These were further

10

undersampled by collapsing two slices that are 60mm apart to simulateMB=2 slice acceleration. The

highly subsampled diffusion msEPI data (RinplanexMB=9x2) were reconstructed using SMS-MUSSELS.

Reconstruction errors with respect to the “fully-sampled” reference data were learned using a deep

network.Similarpatchextractionandaugmentationstepswereperformed.

ReconstructionExperiments

SMS-MUSSELSparameteroptimization:Weexploredthedependenceofthereconstructionperformance

ofSMS-MUSSELSonthek-spacewindowsize𝑟,aswellastherankconstraintenforcedbythenumberof

singularvalues,𝑘.FortheSAGEdataset,weevaluatedtheRMSEmetriconaslicegroupfromatraining

subject,andconsideredarangeofwindowsizes𝑟 ∈ {2,3,4,5,6,7}.Tocontroltherankofthedatamatrix

ℋ(𝒙) which has𝑁;×𝑟×𝑟 columns in an intuitive manner, we varied the “effective number of shots

(𝑁_``)” between 𝑁_`` ∈ {0.75, 1, 1.25, 1.5}. For instance, using a window size 𝑟 = 6 and enforcing𝑁_`` = 1.25 would imply that the number of singular values 𝑘 = 𝑁_``×𝑟×𝑟 = 45 is used during thereconstruction.Thisway,𝑁_``givesusahandleontherankconstraintintermsoftheeffectivenumber

of shotswe allow themsEPI data to have. The optimal parameter setting turned out to be 𝑟 = 5 and𝑁_`` = 1forthe2-shotSAGEreconstructionatRinplanexMB=8x2.Terminationcriterionwaslessthan0.1%

update between image estimates from successive iterations. This analysis is presented in Supporting

InformationFigureS1.

UsingDWIdatasetfromatrainingsubject,asimilaranalysisrevealedthattheoptimalparametersetting

is 𝑟 = 7 and𝑁_`` = 1.25 for a 5-shot reconstruction at RinplanexMB=9x2. Best RMSE was obtained by

terminatingtheSMS-MUSSELSiterationswhentheupdateintheimageestimatebetweeniterationswas

lessthan0.3%.

POCS versus FISTA updates: To improve the convergence rate and image quality of our POCS-like

optimization algorithm for SMS-MUSSELS,wehaveexplored FISTAupdate rule,whichmakesuseof a

combinationof the current andprevious iterates to form thenext image estimate (as detailed in the

Appendix).FISTAwaspreviouslyusedinthecontextofdiffusionmsEPIwithlocallowrankconstraintin

image-space (47). A comparison on one of the SAGE training datasets indicated that FISTA provided

substantial reduction of aliasing/ghosting artifacts as well as RMSE improvement, when other

parameterswereheld constant (𝑟 = 5 and𝑁_`` = 1).As such,wehaveusedFISTA iterations for theremaining SMS-MUSSELS reconstructions reported herein. Convergence analysis is provided in

SupportingInformationFigS11.

11

SAGEreconstruction@RinplanexMB=8x2with2-shots:msEPISAGEdatawereacquiredona4thsubject(not

seen during the training of the network). This acquisition was then reconstructed with SMS-MUSSELS

usingFISTA iterationsandtheoptimizedparametersetting inMatlab,runningonaworkstationwith64

CPUprocessorsand256GBmemory.

SMS-MUSSELS shot images were then processed with the trained U-Net, which allowed improved

estimation of shot-phases using phase cycling. We used 500 iterations and “db4” wavelets in phase

cycling,andsettheregularizationparameterto𝛼=10-5foroptimalRMSE.Havingestimatedthephaseof

each shot, JVC-SENSEwith total variation penalty (𝛽=3·10-4)was used to compute the finalmagnitude

image. For all the remaining experiments, we used these reported parameter values without further

optimization.

SAGEreconstructionwithoutML:ToassessthecontributionoftheMLsteptoSMS-NEATR,weperformed

anadditionalreconstructionwithoutU-Netprocessing.Weusedthesameundersamplingsetupfromthe

first experiment, namely RinplanexMB=8x2 acceleration with 2-shots. Starting from the SMS-MUSSELS

magnitudeestimate𝑚dCeeEfe,weemployedphasecyclingtosolve:

𝑚𝑖𝑛IJ 𝐹&𝐶𝑚dCeeEfe ∙ 𝑒LIJ − 𝑑& +++ 𝛼 𝑊𝜙& / Eq4

Havingobtainedrefinedshot-to-shotphaseestimates𝜙&,wewentontouseJVC-SENSEandarriveatarefinedmagnitude solution. Thisway,we followed the flowchart outlined in Fig. 1, except thatwe by-

passedtheU-Netprocessingstep.

Figure 3RinplanexMB=8x2-fold accelerated SAGE msEPI acquisition with 2-shotsfromatrainingdataset.OneSMSslicegroupandtwoechoesoutofatotaloffivearedepicted.UsingaPOCS-likesolverforSMS-MUSSELSoptimizationledtoresidual aliasing/ghosting artifacts (arrows). FISTA update rules improvedconvergence and image quality of SMS-MUSSELS, and mitigated thesestructurederrors

12

SAGE reconstructionusingBM3D instead ofU-Net:Wehave also explored replacing the deepnetwork

withaconventionaldenoiser,BM3D(48),tohelpimprovetheSMS-MUSSELSoutput.Afterdecomposing

theshotimagesintorealandimaginarycomponents,weprocessedeachoftheseimagesseparately,and

normalizedtheirintensitytobewithin[0,1].WeoptimizedtheBM3Dfilterwidth𝜎forthebestRMSE.The

resulting shot imageswereused to initializephase-cyclingand JVC-SENSE reconstructions, i.e.weagain

followedtheflowchartinFig.1,butreplacedU-NetwithBM3D.

T2andT2*parameterfittingusingSAGEdata:ThefiveechoimagesproducedbySMS-MUSSELSandSMS-

NEATRalgorithmswereusedinaBloch-equationbasedmodelfit(22)toestimateT2andT2*parameter

maps. As supplementary information, we have also explored parameter fitting to the reconstructions

obtainedfromU-NetandBM3Ddenoisers.

DWI reconstruction@ RinplanexMB=9x2with 5-shots: DWI data at six directions were acquired on a 4th

subject (not seenduring the training). Twoof thesedirectionswere reconstructedwith SMS-MUSSELS.

Theseimageswererefinedusingthedeepdiffusionnetwork,andwereprocessedwithphase-cyclingand

JVC-SENSE to compute SMS-NEATR results. In this case, 50 phase-cycling iterations with 𝛼=10-3, andTikhonov regularized JVC-SENSE with 𝛽=10-2 yielded optimal RMSEs. We have also explored BM3D

denoising, again with complex-valued processing for each shot separately, and optimized for the filter

width𝜎.

DWI analysis: Six direction diffusion data reconstructed by the algorithms under comparison were

registeredusingMCFLIRT (49).Diffusiontensor fittingwasperformedusing theDTIFIT function inFSL,

whichalsoproducedfractionalanisotropy(FA)andmeandiffusivity(MD)maps.

RESULTS

SAGE reconstruction @ RinplanexMB=8x2 with 2-shots: The left column of Fig. 4 shows SMS-MUSSELS

reconstructions,whereonlythefirstandlastechoesandroot-sum-of-squareserrormapcalculatedover

the entire 5 echoes are displayed. U-Net processing mitigated some of the noise amplification and

improvedtheRMSEfrom10.8%to8.3%.Startingfromthis,SMS-NEATRwasabletoprovideasmallerror

reduction(8.1%),withsimilarlyhighimagequality.

13

SAGEreconstructionwithoutML:SupportingInformationFigS2demonstratestheeffectofnotusingU-

Netdenoising in theSMS-NEATRpipeline. In thiscase, therewasstill somegain fromrefining theshot-

phaseestimatesusingphase-cyclingandjointparallelimagingreconstruction(RMSEwentfrom10.8%to

9.2%),buttheimprovementoverSMS-MUSSELSwasyethigherwhenMLwasincluded(8.1%).

SAGEreconstructionusingBM3DinsteadofU-Net:UsingaconventionalBM3Ddenoisercouldstillprovide

RMSEreduction (9.3%)but the learnedU-Netmodelwasmoresuccessful in refining theSMS-MUSSELS

output (8.3%). Supporting Information Fig S3 also explores using BM3D to replace U-Net in the SMS-

NEATRflowchart,whichappearedtobeslightlylesseffective(8.4%withBM3Dinitializationversus8.1%

withU-Netjumpstart).

T2 and T2* parameter fitting using SAGE data: Parametermaps from a slice group reconstructed using

SMS-MUSSELSandSMS-NEATRaredepictedinFig.4,correspondingtoan8.3secacquisitionwithwhole-

braincoverageat1x1x3mm3resolution.SMS-NEATRwasabletomitigatenoiseamplificationand image

artifactsmainlyaffectingthemiddleoftheFOVintheSMS-MUSSELSmaps.T2andT2*fitsafterBM3Dand

Figure4SAGEtestdatasetatRinplanexMB=8x2-foldaccelerationusing2-shots.ThefirstandlastechoesareshownforasingleSMSslice group. SMS-MUSSELS with FISTA (left) was successful in reconstructing images despite the high acceleration with 10.8%error.Thebottomrowshowsroot-sum-of-squarescombinationoferrorimagesacrossthefiveechoes.U-NetdenoisingofSMS-MUSSELS reconstruction provided improvement (8.3%,middle), andwas used for initializing SMS-NEATR for additional qualitygain(8.1%,right).

14

U-NetdenoisingarecomparedinSupportingInformationFigS4.U-Netestimatesagainappearedtohave

higherqualitythantheBM3Dresults,andweresimilartotheSMS-NEATRmaps.

Supporting Information Figs S12 and S13 show parameter maps from fully-sampled MUSSELS

reconstructionaswellaserrormapsfromtheacceleratedreconstructions.

DWI reconstruction@RinplanexMB=9x2with 5-shots: Fig. 6 showsDWI slice groups fromonediffusion

direction.TheselowersliceswithpoorB0uniformitywereselectedtodemonstratetheeffectofhighin-

planeaccelerationinavoidingdistortionandvoxelpile-upartifacts.Regardless,SMS-MUSSELSdidsuffer

fromresidualghostingandnoiseamplification in thesedifficult reconstructiontasks.BM3DandU-Net

processinghelpeddenoisethedata,butfailedtoeliminatethestructuredartifacts(indicatedbywhite

arrows).BM3Dresultsappearedover-smooth,whereasU-Netprovidedabettertrade-offbetweenover-

smoothing and denoising. SMS-NEATR did not suffer from over-smoothing, and could mitigate noise

amplification and structured artifacts.We anticipate that RMSE values have contributions from both

noiseandreconstructionerrorsincethegroundtruthdataarealsonoisy.Assuch,RMSEis likelytobe

partiallyindicativeofreconstructionperformance(U-Netconsistentlyhadthebestperformance).

Figure5T2andT2*parametermapsobtainedbyBlochequation fittingto thefive-echo SAGE reconstruction. This 2-shot acquisition at RinplanexMB=8x2-foldacceleration provides whole-brain coverage in 8.3 sec with low geometricdistortion. While SMS-MUSSELS parameter maps appeared noisy (left), theseartifactsweremitigatedintheSMS-NEATRestimates(right).

15

Using complex-valuedormagnitude-basedU-Netprocessing led to similar SMS-NEATR results inDWI

(Supporting Information Fig S10). TV-regularizer could provide further RMSE reduction than Tikhonov

penalty,butthiscameatthecostofsomeover-smoothing(SupportingInformationFigS8).Reducingthe

TV regularization parameter led to comparable RMSEs and image sharpness as ℓ+-penalty using bothcomplex-andmagnitude-valuedU-Netinitialization(SupportingInformationFigS8andS9).

Fig.7showsSMS-NEATRresults fromthesixdirectionacquisition,aswellas theaverageDWI image,

colorFAandMDmaps,andtheroot-sum-of-squarecombinationoferror imagesacrossalldirections.A

similaranalysisispresentedforthefully-sampled,SMS-MUSSELSandU-NetreconstructionsinSupporting

InformationFigsS14–S16.

Figure6AnSMSslicegroupofaseconddiffusiondirectionfromthetestmsEPIacquisitionisshown.SMS-MUSSELSsufferedfromnoiseamplificationandsomestructuredartifacts.BM3DandU-NetcoulddenoisetheSMS-MUSSELSresultatthe potential cost of over-smoothing, while some artifacts persisted (arrows).SMS-NEATR could provide better SNR and image quality without thevulnerabilitytoover-smoothing.

16

DISCUSSION

WepresentedSMS-NEATR,asynergisticMLandphysics-basedreconstructionapproach,thatallowedup

toRnet=8-foldacceleratedmsEPIwithhighimagequality.Thiswasmadepossiblebytakingadvantageof

phase-cycling algorithm, the newly developed SMS-MUSSELS, deep learning, and joint parallel imaging

reconstruction.Our residual CNN learned topredict andmitigate the errors in highly accelerated SMS-

MUSSELS reconstruction,which thenpermittedphase-cycling toestimateshot-to-shotphasevariations.

Including this information as additional sensitivity variations then allowed JVC-SENSE to solve for a

commonmagnitudeimageusingtheentiremultishotk-spacedataandVCconcept.

Partial Fourier sampling was not performed during any of the acquisitions. While this would have

helped achieve shorter TE and higher SNR in DWI, it would not affect the geometric fidelity of the

acquisition.Ourmotivation inemployinghigh in-planeaccelerationrateswas to reducedistortionand

T2*-related blurring, as well as enabling shorter TE. Particularly with the SAGE scan, our aim was to

replace the currently inefficient spin-warp imagingwith themuch faster,msEPI-basedacquisitions for

rapidclinicalimaging,whileminimizingghosting,blurringanddistortionartifactsthatplagueEPI.

CNNs can represent very complicated and non-linear input/output relations.While thismakes them

verypowerful,suchacomplexmappingbetweeninputandoutputcausesthenetworktobedifficultto

characterize.Sinceitsdirectapplicationmayleadtounpredictableerrors,end-to-endCNNreconstruction

inclinical settings is likely to raiseacceptance issues.AML reconstructionapproach thatcanovercome

thisissuewasproposedintheVariationalNetwork(VN)(50)formulation.Thisallowsatransparentdeep

learningreconstructionofacceleratedacquisitionwhereboththekernelweightsandnonlinearactivation

Figure7SMS-NEATRreconstructionforsixdirectiondiffusiondata,aswellasaverageDWI,colorFAandMDmapsandroot-sum-of-squareserroracrossthedirectionsarepresented.

17

functionsarelearntandcanbevisualizedatanylayer.VNalsoutilizessensitivityencodingandenforces

consistencytotheacquiredk-spacedata.Similarly,modelbaseddeeplearning(MoDL) ispowerful in its

ability to combine data consistency and convolutional layers (51). These ideas treat the iterations in

gradient-descent type reconstructions as unrolled networks to retain fidelity to acquired data via a

forwardmodel,whilelearningmodelparametersthatmapthereconstructiontoareferenceimage(52).

Importantly, such combination of a forward-model and learned filtering provided further improvement

thanamodel-basedreconstructionfollowedbyU-Netdenoising(53).

SMS-NEATRalsotaps intothepotentialofCNNwithout treating itasanend-to-endtool,while fully

harnessingtheencodingprovidedbythescannerhardware.WeachievedthisbyusingCNNtoobtainan

interim imagewithminimalartifacts,whileutilizinga rigorousphysics-basedapproach tovalidateand

improveuponthissolutioninthefinalstepofreconstruction.OurgoalinSMS-NEATRistocaptureshot-

to-shotphasevariationsaccurately,sincewhentheyareknown,aJVC-SENSEreconstructionthatsolves

forthemagnitudeimageiscapableofoutperformingalternativeapproaches.Synergisticcombinationof

MLandphysics-basedreconstructionprovedtobepowerful,leadingto~30%RMSEreductionoverour

SMS-MUSSELS implementation (Figs. 4 and 6). In the absence of deep learning initialization, the

subsequentphase cyclingand JVC-SENSE stepsprovideda smaller, <20% improvementover the SMS-

MUSSELSreconstruction(SupportingInformationFig.S2).AconventionalBM3Ddenoiseralsoprovedto

beeffective in jumpstartingSMS-NEATR,but theperformancewas consistentlybetterusinga learned

denoiser tailored for the specific application (Figs. 6, Supporting Information Figs. S3 and S4). We

anticipatefurthergainsfromadvancedmodelsthatcouldsimultaneouslyenforcedataconsistencyand

perform learned filtering (54–56). Thiswouldalso streamline theSMS-NEATRpipelineand reduce the

numberofsteps.

ApplicationofmsEPI instructural imagingismadedifficultbyghostingartifactsfromhard-to-estimate

physiologicsignalchangesbetweenshots.ThisisparticularlytrueforgradientechoimagingatlateTEs.To

illustrate,we performed a “slidingwindow” combination of 8-shots of SAGE data acquired at Rinplane=8

acceleration toobtain “fully-sampled”data (Supporting Information Fig. S5). The ghosting artifacts that

stemfromphysiologicalnoiseisespeciallystronginthe2ndand3rdechoesduetoincreasedphaseaccrual

atlongTEs(thelastechoisinfactaspinecho,whichrefocusesmostofthephaseevolutionandresultsin

the cleanest image). Using a standard forward-model based reconstruction for structuralmsEPI would

thusnecessitatethesimultaneousestimationoftheimagecontentandthephasevariationsineachshot.

Since both the clean image and the phase information used in the forward-model are unknown, this

wouldentail the solutionof a computationallyprohibitive,non-convexoptimizationproblem that could

get stuck at localminima. As such, existingmsEPI techniques circumvent this difficulty by dividing the

reconstruction into two separate parts: shot-phase estimation, and combination ofmultishot given the

estimatedphaseinformation.Navigator-basedapproachesderivethisphaseinformationfromadditional

calibration acquisitions made for each shot (3–7). Diffusion imaging with MUSE and its extensions

(9,12,13)operatewithoutanavigator, andperform thephaseestimation stepusingparallel imaging to

18

reconstructacompleximageforeachshot.Smoothingthephaseofeachintermediateimagethenyields

anestimateofshot-to-shotvariations,whichallowsjointreconstructionofallmultishotdatatogether.

msEPI reconstruction is indeed harder in diffusion imaging, since the phase variations amplified by

diffusion gradients can be much stronger than the physiologic noise in structural imaging, as

demonstrated in the “sliding window” combination in Supporting Information Fig. S6. The final

Supporting InformationFig.S7showsasimilarslidingwindowdatacombination,butthistimewithout

any diffusion gradients (b=0). Even in this case where one would not expect any artifacts, there are

minor ghosts that may be stemming from patient motion. Given that the TR was 5.1sec, msEPI

acquisitionsare indeedsusceptible tomotionartifacts since it took~46sec tosample these9-shots.A

sidebenefitofhighlyacceleratedmsEPIcouldbeanimprovementinmotionrobustness.Wehaveseen

similar gains in the final SMS-NEATR diffusion reconstructions using either magnitude- or complex-

valueddeeplearning.Weexpectthiswasbecausethemagnitudenetworkcouldprovidehigherquality

magnitude priors which helped phase-cycling to better solve for the shot-phase data, whereas the

complexnetworkprovidedanoverallgaininbothmagnitudeandphaseestimates–butthemagnitude

outputwasimprovedtoalesserextent(ascanbeseenintheRMSEvaluesinSupportingInformationFig

S10).HavingobtainedsimilarSMS-NEATRresultsmayindicatethatthereisflexibilityintheblocksinthe

pipeline,aslongastheshot-phaseestimatesareimprovedbeyondthoseofSMS-MUSSELS.

MUSSELS exploits similarities between the shot-images using a low-rank prior on the block-Hankel

representationoftheirk-space(11,14),sothatitcanperformmsEPIreconstructionwithoutexplicitshot-

phase estimation. MUSSELS has allowed Rinplane=8-fold acceleration per each shot in msEPI diffusion

imagingusingasfewas4-shots(Rnet=2).Unlikeearliernavigator-based(5–7)ornavigator-freeapproaches

(8–10) where the number of acquired shots was equal to the in-plane acceleration factor (𝑁;=Rinplane),

MUSSELS could thus perform in the (𝑁;<Rinplane) regime to improve acquisition efficiency. With SMS-

NEATR,wepushedtheefficiencygainevenfurthertoenableRnet=8-foldacceleration(RinplanexMB=8x2with

2-shots) in structural imaging, and Rnet=3.6-fold (RinplanexMB=9x2 with 5-shots) in diffusion imaging.

AlthoughSMS-MUSSELShadsomeresidualartifactsatsuchhighaccelerations, itprovidedagood initial

guess for our residual network to further clean up the shot-images. Starting from these estimates,we

couldthensolveforthephasevariationsusingphasecycling,whichconstitutedaneasierproblemsince

theunknowninformationwasareal-valuedphaseimage.Thisprovideda2-foldreductioninthenumber

ofunknownscomparedtoacomplex-valuedSENSEsolution.

Thebest RMSEperformance for structural imagingwasobtainedwith𝑟×𝑟 = 5×5windows andwith𝑁_``=1,whereas the optimal parameterswere𝑟×𝑟 = 7×7windows andwith𝑁_``=1.25 for DWI. The

increasedwindowsizeandrankconstraintshouldhelpcapturegreatershot-to-shotvariations,whichare

morelikelytobeobservedindiffusionimagingthantheSAGEscan.Furtherrelaxingtherankconstraint

and using larger windows could help representmore spatially varying phases differences between the

shots, but relaxing thesepriorsbeyond their optimal valueswould comeat thepotential costofRMSE

19

performance. Indeed, using𝑁_`` = 𝑁; would be a non-informative prior and the outcome would be

identicaltoashot-by-shotSENSEreconstruction.

Limitations and theirmitigation: SMS-NEATRusesML to provide an initial estimate to a difficult image

reconstruction problem, thereby avoiding the vulnerability of poor generalization of “direct” ML

reconstruction.Inadditiontothis,wehaveaugmentedourtrainingdatasetsizeby16-foldtosubjectthe

networktogreatervariation.Usingapatch-wiserepresentationwithoverlappingpatcheshelpedfurther

increasetheavailablenumberof trainingsamples.Finally,wehaveprovidedthenetworkwithdifferent

contrasts (echoes or diffusion directions) as training samples to help improve generalization. Despite

theseprecautions, thenetworkwouldbenefit fromre-training if largechanges in sequenceparameters

are desired to be made, or if they are dictated by hardware limitations of other scanners. We also

anticipate that having the subsequent physics-based reconstruction will mitigate some of the

generalizationconcerns,astheMLoutputisusedforinitializingthismodelbasedstep.Exploringunrolled

networks with data consistency layers (50–53,55) or using conventional denoisers could provide

additional robustness.Using smoothness priors embedded in theMUSSELS reconstructionwith the SR-

MUSSELS formalism rather than relyingon learnedor conventional denoiserswouldalsobeanelegant

solution.

Anotherconsiderationistheselectionofthereconstructiontechniquethatprovidestheinitialsolution

toU-Net.WehavedevelopedaFISTA-basedsolverforSMS-MUSSELS,butotheradvancedreconstruction

strategiessuchasMUSE(9)orPOCS-MUSE(10)couldalsobeutilizedtoprovidethisinitialestimate.

Qualitatively, SMS-NEATR provided greater gains in the more challenging multishot DWI

reconstruction than the SAGE application. It has bettermitigated ghosting/aliasing artifacts and noise

amplification than SMS-MUSSELS, but the RMSE metrics remained above 20%. We think that this is

becausethegroundtruthdiffusiondataisalsocorruptedbynoise,whichmakesitdifficulttodisentangle

reconstruction artifacts from the noise contribution. As such, othermeasures of fidelity to reference

datacouldbettergaugetheimprovementinDWIreconstruction.

For Nyquist ghost correction, we have used a simple 1-dimensional navigator in a slice-specificmanner. Especially in oblique acquisitions, more involved ghost correction techniques such as DualPolarityGrappa(57)andLORAKS(31)shouldallowforimprovedsuppressionoftheseerrors.Toensurethat the residual ghosts seen in presented results stem only from reconstruction errors due toacceleration,wehave includedfully-sampledMUSSELSreconstructionsandFLEETcalibrationdatathatdonotexhibitghostingintheSupportingInformationFigsS17andS18.

Extensions:WehavedemonstratedtheapplicationsofSMS-NEATRinmsEPISAGEandDWIacquisitions.

EnablingRinplane=8or9-foldaccelerationinotherpulsesequencescouldhelpcreateamulti-contrastmsEPI

clinicalprotocolwithhighgeometricfidelity.Thiswouldminimizethedistortionandblurringartifactsthat

hamperimagequalityandachievableresolutionintherecentlydevelopedsingleshotEPIprotocols(1,2).

Employing msEPI readout in multi-inversion T1 mapping (58,59) and FLAIR (60) acquisitions with SMS-

NEATRreconstructioncouldenablearapidMRexamwithsimilartabletimeasaCTscan.Otheradvanced

20

encoding strategies such as wave-EPI (61) could provide additional efficiency gain and/or in-plane

accelerationcapability.

WebelievethatthestrategyofutilizingMLtoestimateunknownnuisanceparametersinphysics-based

forwardmodelreconstructionscanbeimpactfulinsolvingotherprohibitivelydifficultproblems.Wehave

recentlydemonstratedthisconceptinprospectivemotioncorrection(62),whereweusedresidualdeep

learning to provide an interim image with largely reduced motion artifacts. This interim CNN

reconstructionprovides an initial image andmotionparameter estimate thus jumpstarting thephysics-

based TAMER algorithm (63), which uses the extra degrees of freedom in multi-coil data to jointly

estimatemotionparametersandtheclean image.Havingaccesstoagoodinitialguesshelpedthenon-

convexTAMERoptimizationconverge30× faster to the final solution. Othervenues thatmightbenefit

from this synergistic approach couldbe innavigator-freeNyquist (N/2) ghost correction, calibrationless

parallelimagingandreference-freek-spacetrajectoryestimation.

CONCLUSION

WedemonstratedtheabilityofSMS-NEATR,acombinedMLandphysics-basedreconstructionalgorithm,

inprovidinghighqualityreconstructionsfromupto8-foldacceleratedmsEPIacquisitionsusing2–5shots

ofdata.Theabilitytoacquirehighin-planeresolutionimageswithminimaldistortionandblurringcould

enableanmsEPI-basedMRIexamwithmultiplecontrasts,whilematchingthetabletimeofaCTscan.

ACKNOWLEDGMENT

We acknowledge a GPU grant fromNVIDIA, and support fromNIH NINDS (K23 NS096056) NIMH (R24

MH106096andR01MH116173)andNIBIB(U01EB025162,R01EB020613,R01EB019437,R01EB017337

andP41EB015896).AdditionalsupportwasprovidedbytheMGH/HSTAthinoulaA.MartinosCenterfor

Biomedical Imaging. This research was made possible by the resources provided by NIH Shared

InstrumentationGrantsS10-RR023401andS10-RR023043.

APPENDIX

WepursueaPOCS-likesolutiontotheoptimizationproblemposedintheMUSSELSformalism(Eq1),and

followthestepsbelow:

21

𝒚𝟏 = 𝒙𝟎%initialguessfrome.g.SMS-SENSEreconstruction

𝜏/ = 1for𝑖 = 1: 𝑁L&_m

%Low-rankconstraint:

𝐴 = ℋ(𝒚𝒊)𝑈Σ𝑉s = 𝑠𝑣𝑑(𝐴)𝐴 = 𝑈Σv𝑉s %Σvisobtainedviahardthresholdingbykeepingthe𝑘largestsingularvalues𝒙𝒊 = ℋ∗ 𝐴 %ℋ∗isatransposedmappingthatinsertsHankelmatrixelementsintomulti-shotk-space

for𝑡 = 1: 𝑁;𝑥& = 𝒙𝒊(: , : , 𝑡)%Generatecoilimagesbymultiplicationwithsensitivities:

𝑥w = 𝐶𝑥&%Resubstituteacquiredk-space:

𝑥w = 𝑥w + 𝐹s(𝑑& − 𝐹&𝑥w)%Coilcombination:

𝑥& = 𝐶s𝐶 Q/𝐶s𝑥w 𝒙𝒊 : , : , 𝑡 = 𝑥&

end

ifuse_fista

𝜏Lx/ =1 + 1 + 4𝜏L+

2

𝒚𝒊x𝟏 = 𝒙𝒊 +𝜏L − 1𝜏Lx/

𝒙𝒊 − 𝒙𝒊Q𝟏

else

𝒚𝒊x𝟏 = 𝒙𝒊end

end

The flag “use_fista” togglesbetweenconventionalPOCS-likeupdate ruleandFISTA iteration,whichhas

earlieriteratestoformthenextimageestimate.𝑁L&_m denotesthemaximumnumberofiterations,which

wehavetakentobe200.𝒙𝟎 isaninitialguessforthemsEPIimages,andwereestimatedusinganSMS-

SENSEreconstructionforeachoftheshotsindependently.

REFERENCES

1.SkareS,SprengerT,NorbeckO,RydénH,BlombergL,AvventiE,EngströmM.A1-minutefullbrainMRexamusingamulticontrastEPIsequence.Magn.Reson.Med.2017;00:1–10.doi:10.1002/mrm.26974.

22

2.NaelK,KhanR,ChoudharyG,MeshksarA,VillablancaP,TayJ,DrakeK,CoullBM,KidwellCS.Six-minutemagnetic resonance imaging protocol for evaluation of acute ischemic stroke: Pushing the boundaries.Stroke2014;45:1985–1991.doi:10.1161/STROKEAHA.114.005305.

3.PorterDA,HeidemannRM.Highresolutiondiffusion-weightedimagingusingreadout-segmentedecho-planarimaging,parallelimagingandatwo-dimensionalnavigator-basedreacquisition.Magn.Reson.Med.2009;62:468–475.doi:10.1002/mrm.22024.

4.JeongH-K,GoreJC,AndersonAW.High-resolutionhumandiffusiontensorimagingusing2-DnavigatedmultishotSENSEEPIat7T.Magn.Reson.Med.2013;69:793–802.doi:10.1002/mrm.24320.

5. LiuW, ZhaoX,Ma Y, Tang X,Gao J-H.DWIusingnavigated interleavedmultishot EPIwith realignedGRAPPAreconstruction.Magn.Reson.Med.2016;75:280–286.

6.Ma X, Zhang Z, Dai E, Guo H. Improvedmulti-shot diffusion imaging using GRAPPAwith a compactkernel.Neuroimage2016;138:88–99.doi:10.1016/J.NEUROIMAGE.2016.05.079.

7.DongZ,WangF,MaX,ZhangZ,DaiE,YuanC,GuoH. InterleavedEPIdiffusion imagingusingSPIRiT-based reconstruction with virtual coil compression. Magn. Reson. Med. 2018;79:1525–1531. doi:10.1002/mrm.26768.

8.ZhangZ,HuangF,MaX,XieS,GuoH.Self-feedingMUSE:ArobustmethodforhighresolutiondiffusionimagingusinginterleavedEPI.Neuroimage2015;105:552–560.doi:10.1016/J.NEUROIMAGE.2014.10.022.

9.ChenNK,GuidonA,ChangHC,SongAW.Arobustmulti-shotscanstrategyforhigh-resolutiondiffusionweighted MRI enabled by multiplexed sensitivity-encoding (MUSE). Neuroimage 2013;72:41–47. doi:10.1016/J.NEUROIMAGE.2013.01.038.

10. Chu M-L, Chang H-C, Chung H-W, Truong T-K, Bashir MR, Chen N. POCS-based reconstruction ofmultiplexed sensitivity encoded MRI (POCSMUSE): A general algorithm for reducing motion-relatedartifacts.Magn.Reson.Med.2015;74:1336–1348.doi:10.1002/mrm.25527.

11.ManiM,JacobM,KelleyD,MagnottaV.Multi-shotsensitivity-encodeddiffusiondatarecoveryusingstructured low-rank matrix completion (MUSSELS). Magn. Reson. Med. 2017;78:494–507. doi:10.1002/mrm.26382.

12. Zhang Z, Huang F, Ma X, Xie S, Guo H. Self-feeding MUSE: A robust method for high resolutiondiffusion imaging using interleaved EPI. Neuroimage 2015;105:552–560. doi:10.1016/J.NEUROIMAGE.2014.10.022.

13. Chu M-L, Chang H-C, Chung H-W, Truong T-K, Bashir MR, Chen N. POCS-based reconstruction ofmultiplexed sensitivity encoded MRI (POCSMUSE): A general algorithm for reducing motion-relatedartifacts.Magn.Reson.Med.2015;74:1336–1348.doi:10.1002/mrm.25527.

14.ManiM,MagnottaV,KelleyD, JacobM.Comprehensive reconstructionofmulti-shotmulti-channeldiffusion data usingmussels. In: 2016 38thAnnual International Conference of the IEEE Engineering inMedicineandBiologySociety(EMBC).IEEE;2016.pp.1107–1110.doi:10.1109/EMBC.2016.7590897.

15.LiuC,MoseleyME,BammerR.SimultaneousphasecorrectionandSENSEreconstructionfornavigatedmulti-shot DWI with non-cartesian k-space sampling. Magn. Reson. Med. 2005;54:1412–1422. doi:10.1002/mrm.20706.

23

16.PipeJG,FarthingVG,ForbesKP.Multishotdiffusion-weightedFSEusingPROPELLERMRI.Magn.Reson.Med.2002;47:42–52.doi:10.1002/mrm.10014.

17.KoopmansPJ,PoserBA,BreuerFA.2D-SENSE-GRAPPAforFast,Ghosting-RobustReconstructionofIn-Plane and Slice Accelerated Blipped-CAIPI-EPI. In: International Society for Magnetic Resonance inMedicine23rdScientificMeeting,Toronto,Canada.;2015.p.2410.

18.OngF,ChengJ,LustigM.SupportingInformationfor :GeneralPhaseRegularizedReconstructionusingPhaseCycling.2017:1–3.

19. Blaimer M, Gutberlet M, Kellman P, Breuer F, Kostler H, Griswold MA. Virtual coil concept forimprovedparallelMRIemployingconjugatesymmetricsignals.Magn.Reson.Med.2009;61:93–102.

20. Blaimer M, Heim M, Neumann D, Jakob P, Kannengiesser SA, Breuer F. Comparison of phase-constrainedparallelMRIapproaches:Analogiesanddifferences.Magn.Reson.Med.2016;75:1086–1099.

21.BlaimerM,JakobP,BreuerF.Regularizationmethodforphase-constrainedparallelMRI.Magn.Reson.Med.2014;72.1:166–171.

22.SchmiedeskampH,StrakaM,NewbouldRD,ZaharchukG,Andre JB,Olivot J-M,MoseleyME,AlbersGW, Bammer R. Combined spin- and gradient-echo perfusion-weighted imaging. Magn. Reson. Med.2012;68:30–40.doi:10.1002/mrm.23195.

23.RonnebergerO,FischerP,BroxT.U-Net:ConvolutionalNetworksforBiomedicalImageSegmentation.In: Medical image computing and computer-assisted intervention : MICCAI. Springer, Cham; 2015. pp.234–241.doi:10.1007/978-3-319-24574-4_28.

24.OngF,ChengJY,LustigM.Generalphaseregularizedreconstructionusingphasecycling.Magn.Reson.Med.2018;80:112–125.doi:10.1002/mrm.27011.

25. Pruessmann KP,Weiger M, Scheidegger MB, Boesiger P. SENSE: Sensitivity encoding for fast MRI.Magn.Reson.Med.1999;42:952–962.

26.Haldar JP. Low-RankModeling of Local k-SpaceNeighborhoods (LORAKS) for ConstrainedMRI. IEEETrans.Med.Imaging2014;33:668–681.doi:10.1109/TMI.2013.2293974.

27.JinKH,YeJC.Sparse+LowRankDecompositionofAnnihilatingFilter-basedHankelMatrixforImpulseNoiseRemoval.2015.

28. Shin PJ, Larson PEZ, OhligerMA, EladM, Pauly JM, Vigneron DB, LustigM. Calibrationless parallelimaging reconstruction based on structured low-rank matrix completion. Magn. Reson. Med.2014;72:959–970.doi:10.1002/mrm.24997.

29. Samsonov AA, Kholmovski EG, Parker DL, Johnson CR. POCSENSE: POCS-based reconstruction forsensitivity encoded magnetic resonance imaging. Magn. Reson. Med. 2004;52:1397–1406. doi:10.1002/mrm.20285.

30. Beck A, TeboulleM. A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems.Soc.Ind.Appl.Math.2009;2:183–202.doi:10.1137/080716542.

31.LobosRA,KimTH,HogeWS,HaldarJP.Navigator-FreeEPIGhostCorrectionWithStructuredLow-RankMatrix Models: New Theory and Methods. IEEE Trans. Med. Imaging 2018;37:2390–2402. doi:10.1109/TMI.2018.2822053.

24

32.ZhangK,ZuoW,ChenY,MengD,ZhangL.BeyondaGaussiandenoiser:ResiduallearningofdeepCNNforimagedenoising.IEEETrans.ImageProcess.2017;26:3142–3155.doi:10.1109/TIP.2017.2662206.

33.HeK,ZhangX,RenS,SunJ.DeepResidualLearningforImageRecognition.In:ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition.;2016.pp.770–778.

34.SrivastavaN,HintonG,KrizhevskyA,SutskeverI,SalakhutdinovR.Dropout:ASimpleWaytoPreventNeuralNetworksfromOverfitting.J.Mach.Learn.Res.2014;15:1929–1958.

35.MaasAL,HannunAY,NgAY.Rectifiernonlinearities improveneuralnetworkacousticmodels. ICMLWork.DeepLearn.AUDIO,SPEECHLang.Process.2013;30:3.

36. Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internalcovariateshift.Int.Conf.Mach.Learn.2015:448–456.

37.CholletF.https://keras.io.2015.

38.AbadiM,AgarwalA,BarhamP, et al. TensorFlow: Large-ScaleMachine LearningonHeterogeneousDistributedSystems.2016.

39.KingmaD,BaJ.Adam:Amethodforstochasticoptimization.arXivPrepr.2014:arXiv:1412.6980.

40.BilgicB,Chatnuntawech I,ManhardMK, TianQ, LiaoC,Cauley SF,HuangSY, Polimeni JR,Wald LL,Setsompop K. Highly Accelerated Multishot EPI through Synergistic Machine Learning and JointReconstruction.2018:ArXiv:1808.02814.

41. Bilgic B, Chatnuntawech I,ManhardM, et al.NEATR-SMS forHighly AcceleratedMulti-Shot EPI. In:ISMRMWorkshoponMachineLearningPart2.;2018.

42. Aggarwal HK, ManiMP, JacobM.Multi-Shot Sensitivity-Encoded DiffusionMRI usingModel-BasedDeepLearning(MODL-MUSSELS).2018:ArXiv:1812.08115.

43.ManiM,JacobM,McKinnonG,YangB,RuttB,KerrA,MagnottaV.SMSMUSSELS:ANavigator-freeReconstruction for Simultaneous MultiSlice Accelerated MultiShot Diffusion Weighted Imaging.2019:ArXiv:1901.01893.

44. Uecker M, Lai P, Murphy MJ, Virtue P, Elad M, Pauly JM, Vasanawala SS, Lustig M. ESPIRiT-aneigenvalue approach to autocalibrating parallelMRI:Where SENSEmeetsGRAPPA.Magn. Reson.Med.2014;71:990–1001.doi:10.1002/mrm.24751.

45. Uecker M, Ong F, Tamir JI, Bahri D, Virtue P, Cheng JY, Zhang T, Lustig M. Berkeley AdvancedReconstructionToolbox.In:ProceedingsoftheInternationalSocietyforMagneticResonanceinMedicine.;2015.p.2486.

46.PolimeniJR,BhatH,WitzelT,BennerT,FeiweierT,InatiSJ,RenvallV,HeberleinK,WaldLL.Reducingsensitivity losses due to respiration and motion in accelerated echo planar imaging by reordering theautocalibrationdataacquisition.Magn.Reson.Med.2016;75:665–679.doi:10.1002/mrm.25628.

47. Hu Y, Levine EG, Tian Q, Moran CJ, Wang X, Taviani V, Vasanawala SS, McNab JA, Daniel BA,Hargreaves BL. Motion-robust reconstruction of multishot diffusion-weighted images without phaseestimationthroughlocallylow-rankregularization.Magn.Reson.Med.2018.doi:10.1002/mrm.27488.

48. Dabov K, Foi A, Katkovnik V, Egiazarian K. Image Denoising by Sparse 3-D Transform-Domain

25

CollaborativeFiltering.IEEETrans.ImageProcess.2007;16:2080–2095.doi:10.1109/TIP.2007.901238.

49. JenkinsonM,BannisterPR,BradyM,SmithSM. ImprovedOptimizationfortheRobustandAccurateLinear Registration and Motion Correction of Brain Images. Neuroimage 2002;17:825–841. doi:10.1006/NIMG.2002.1132.

50. Hammernik K, Klatzer T, Kobler E, RechtMP, Sodickson DK, Pock T, Knoll F. Learning a variationalnetwork for reconstruction of accelerated MRI data. Magn. Reson. Med. 2018;79:3055–3071. doi:10.1002/mrm.26977.

51.AggarwalH,ManiM,JacobM.MoDL:ModelBasedDeepLearningArchitectureforInverseProblems.IEEETrans.Med.Imaging2018:1–1.doi:10.1109/TMI.2018.2865356.

52. Adler J, ÖktemO. Solving ill-posed inverse problems using iterative deep neural networks. InverseProbl.2017;33:124007.doi:10.1088/1361-6420/aa9581.

53.AdlerJ,ÖktemO.LearnedPrimal-dualReconstruction.arXiv:1707.06474[math.OC]2017.

54.LeeJ,HanY,YeJC.k-SpaceDeepLearningforReference-freeEPIGhostCorrection.arxiv:1806.001532018.

55.HanY,YeJC.k-SpaceDeepLearningforAcceleratedMRI.arxiv:1805.037792018.

56.AdlerJ,OktemO.LearnedPrimal-DualReconstruction.IEEETrans.Med.Imaging2018;37:1322–1332.doi:10.1109/TMI.2018.2799231.

57.HogeWS,PolimeniJR.Dual-polarityGRAPPAforsimultaneousreconstructionandghostcorrectionofechoplanarimagingdata.Magn.Reson.Med.2016;76:32–44.doi:10.1002/mrm.25839.

58.RenvallV,WitzelT,WaldLL,PolimeniJR.Automaticcorticalsurfacereconstructionofhigh-resolutionT1echoplanarimagingdata.Neuroimage2016;134:338–354.doi:10.1016/J.NEUROIMAGE.2016.04.004.

59.CohenO,Polimeni JR.Optimized inversion-timeschedules forquantitativeT1measurementsbasedonhigh-resolutionmulti-inversionEPI.Magn.Reson.Med.2017.doi:10.1002/mrm.26889.

60.DeCoeneB,HajnalJV.,GatehouseP,LongmoreD,WhiteSJ,OatridgeA,PennockJM,Young ianR,BydderGM.MRof theBrainUsingFluid-Attenuated InversionRecovery (FLAIR)PulseSequences.Am. J.Neuroradiol.1992;13:1555–1564.

61.PoserBA,BilgicB,GagoskiBA,UludagK,StengerVA,WaldLL,SetsompopK.Echo-planarimagingwithwave-CAIPIacquisitionandreconstruction. In: InternationalSocietyforMagneticResonanceinMedicine25thScientificMeeting,Hawaii.;2017.p.1198.

62.BilgicB,CauleySF,ChatnuntawechI,ManhardMK,WangF,HaskellM,LiaoC,WaldLL,SetsompopK.CombiningMRPhysicsandMachineLearningtoTackleIntractableProblems.In:Proceedingsofthe26thAnnualMeetingISMRM.;2018.

63. Haskell MW, Cauley SF, Wald LL. TArgeted Motion Estimation and Reduction (TAMER): DataConsistencyBasedMotionMitigationforMRIusingaReducedModelJointOptimization.IEEETrans.Med.Imaging2018:1–1.doi:10.1109/TMI.2018.2791482.

26

FIGURECAPTIONS

Fig1.SMS-NEATRisacombinedmachinelearningandphysics-basedreconstructiontechniqueforhighly-

acceleratedmsEPIacquisition.WedevelopedSMS-MUSSELSalgorithmtoprovideaninitialsolution,which

may suffer from artifacts due to high acceleration (RinplanexMB=8x2 with 2-shots). Starting from this,

residual learning with U-Net architecture provides an interim image with minimal artifacts. Given this

solution,phasecyclingalgorithmisusedforestimatingshot-phases,whicharethenutilizedassensitivity

variationsinafinaljointvirtualcoil(JVC)SENSEreconstruction.

Fig 2. U-Net architecture is used to learn themapping between patches of shot-images reconstructed

with SMS-MUSSELS, and their difference to reference data. Both the input and output have been

decomposed into real and imaginary components to enable complex-valued processing for SAGE

reconstruction.64×64patchesfromalltheshotsarepresentedasinputtoa5-levelnetwork,wherethe

first level uses 64 convolutional filters. To help provide scale invariance, max pooling operators

downsamplethepatchesaftereachlayer.Atthesametime,thenumberoffiltersaredoubledtoretain

thetotalnumberofkernelweightsateachlevel.

Fig3.RinplanexMB=8x2-foldacceleratedSAGEmsEPIacquisitionwith2-shots fromatrainingdataset.One

SMS slice group and two echoes out of a total of five are depicted. Using a POCS-like solver for SMS-

MUSSELS optimization led to residual aliasing/ghosting artifacts (arrows). FISTA update rules improved

convergenceandimagequalityofSMS-MUSSELS,andmitigatedthesestructurederrors.

Fig4.SAGE testdatasetatRinplanexMB=8x2-foldaccelerationusing2-shots.The firstand lastechoesare

shown for a single SMS slice group. SMS-MUSSELS with FISTA (left) was successful in reconstructing

images despite the high acceleration with 10.8% error. The bottom row shows root-sum-of-squares

combination of error images across the five echoes. U-Net denoising of SMS-MUSSELS reconstruction

providedimprovement(8.3%,middle),andwasusedforinitializingSMS-NEATRforadditionalqualitygain

(8.1%,right).

Fig5.T2andT2*parametermapsobtainedbyBlochequationfittingtothefive-echoSAGEreconstruction.

This2-shotacquisitionatRinplanexMB=8x2-foldaccelerationprovideswhole-braincoveragein8.3secwith

lowgeometricdistortion.WhileSMS-MUSSELSparametermapsappearednoisy(left),theseartifactswere

mitigatedintheSMS-NEATRestimates(right).

Fig6.DiffusionmsEPIacquisitionatRinplanexMB=9x2accelerationwith5-shotsfromthetestsubject.One

SMSslicegroupisshownforthiswhole-brainacquisition.SMS-MUSSELSsufferedfromaliasing/ghosting

artifactsinthisharderreconstructionproblem.BM3DandU-Netdenoisingcouldmitigatenoise,butthe

structuredartifactspersisted(arrows).SMS-NEATRwasabletofurthermitigatetheseerrorsto improve

imagequality,whileavoidingpotentialover-smoothingBM3DandU-Netmaysufferfrom.

Fig7.SMS-NEATRreconstructionforsixdirectiondiffusiondata,aswellasaverageDWI,colorFAandMD

mapsandroot-sum-of-squareserroracrossthedirectionsarepresented.

27

SUPPORTINGINFORMATIONFIGURECAPTIONS

SupportingInformationFigS1.DependenceofthereconstructionperformanceofSMS-MUSSELSonthek-

spacewindowsize𝑟,andtherankconstraintasrepresentedbytheeffectivenumberofshots(𝑁_``)foraslicegroupfromaSAGEtrainingdataset.Theoptimalparametersettingwas𝑟 = 5and𝑁_`` = 1forthe2-shotSAGEreconstructionatRinplanexMB=8x2acceleration.

SupportingInformationFigS2.SAGEmsEPIreconstructionresultsfromthetestdatasetatRinplanexMB=8x2

accelerationusing2-shots.Thefirstandlastechoesaredisplayedoutofatotaloffiveechoesbelongingto

this SMS slice group. SMS-MUSSELS yielded 10.8% RMSE (left), and was also used to initialize phase-

cycling and JVC-SENSE reconstruction without machine learning (9.2% error, middle). Using U-Net to

refinetheSMS-MUSSELSresultandjumpstartSMS-NEATRprovidedfurtherimprovementat8.1%RMSE.

Supporting Information Fig S3. The SMS-MUSSELS reconstruction for the slice group in Fig S2 was

denoisedusingBM3DandU-Net,wheredeeplearningprovedtobeadvantageous(BM3D:9.3%versusU-

Net: 8.3% error). Utilizing each of these denoised outputs to jumpstart SMS-NEATR led to similar

reconstructions,andU-Netinitializationhadthebestoverallperformance(8.1%RMSE).

Supporting Information Fig S4. Bloch equation based signal modeling for the five echoes in the SAGE

acquisitionallowsforT2andT2*parametermapping.Denoisingthe2-shotSMS-MUSSELSreconstruction

atRinplanexMB=8x2acceleration,corresponding toan8.3secwhole-brainacquisition,usingBM3DandU-

Net led to improvements in thequalityof thesequantitativemaps.U-Netappearedmore successful in

mitigatingthenoiseamplificationinthemiddleoftheFOVthantheconventionalBM3Dfiltering.

SupportingInformationFigS5.Slidingwindow(summationacrossshotsink-space)combinationof8-shots

of SAGE data acquired at Rinplane=8 acceleration per shot. Due to physiological shot-to-shot phase

variations, the combined fully-encoded images exhibit ghosting artifacts. This becomesmore severe at

laterTEs,but ismitigatedat the lastecho,which isa spinecho image.Thebottomrow is scaledup to

betterdemonstratetheartifacts.

Supporting InformationFigS6.Slidingwindowcombinationof9-shotsofDWIdataacquiredatRinplane=9

accelerationpershot.Fiveslicesfromawhole-brainacquisitionaredepicted.Duetoshot-to-shotphase

variations stemming from motion under the diffusion encoding gradients, there are severe ghosting

artifactsintheseotherwisefully-encodedimages.

SupportingInformationFigS7.Slidingwindowcombinationofdatafromthesameacquisitionsessionas

FigS6,but this time thediffusiongradientshavebeenswitchedoff (b=0).Thesespinecho images look

relativelydevoidofghostingartifacts,butthescaled-upimagesinthebottomrowrevealthattheseerrors

persist.We anticipate that headmotion contributed to these artifacts, since it took around 46 sec to

sample9-shotsatTR=5.1sec.

28

Supporting Information Fig S8. Employing TV-regularization in JVC-SENSE led to similar results as L2

penalty.ReconstructionobtainedwiththeTVparametervaluethatyieldedtheoptimalRMSEvalue(𝜆z{ =

10Q+)appearedover-smooth,hencereducingtheregularizationto𝜆z{ = 3 ∙ 10Q|providedabettertrade-

offbetweenRMSEperformanceandimagesharpness.Magnitude-basedU-NetwasusedtoinitializeSMS-

NEATRinthesereconstructions.

SupportingInformationFigS9.Usingcomplex-valueddeeplearningtoinitializeJVC-SENSEyieldedsimilar

quality reconstructions asmagnitude-based U-Net processing. JVC-SENSE could flexibly utilize L2 or TV

regularizerstofurtherstabilizethereconstructionwithcomparableresults.

Supporting Information Fig S10. Both complex- andmagnitude-valued U-Net processing could denoise

diffusionimages,albeitatthecostofremainingartifacts(arrows)andespeciallyinthecaseofmagnitude-

valuednetwork,over-smoothing.UsingthesetojumpstartSMS-NEATRledtocrispimageswithmitigated

artifacts.

Supporting Information Fig S11. FISTA update rule helped stabilize SMS-MUSSELS reconstruction,

especially in later iterationswherePOCS still experienced large signalupdates.Different colors indicate

different echo images. Termination criterion was reaching less than 0.1% change between successive

imageestimatesandmaximumiterationnumberwas200.

Supporting Information Fig S12. Parameter maps from fully-sampled MUSSELS reconstruction,

correspondingtoan8-shot,66.4secondSAGEscan.

Supporting InformationFigS13.Parametererrormaps fromRinplane xSMS=8x2-foldacceleratedSMS-

MUSSELS,U-NetandSMS-NEATRreconstructionsrelativetothefully-sampleddata.

Supporting Information Fig S14. Diffusion images from 6-directions, average diffusion weighted image

(DWI),b=0,colorfractionalanisotropy(FA)andmeandiffusivity(MD)mapsfromthefully-sampledmsEPI

acquisitionwithMUSSELSreconstruction.

Supporting Information Fig S15. Diffusion images from 6-directions, root-sum-of-squares (RSoS)

combinationoferroracrossthe6-directions,averageDWI,colorFAandMDmapsfromtheaccelerated

SMS-MUSSELSreconstruction.

SupportingInformationFigS16.6-directiondiffusionimages,RSoSerroracrossthe6-directions,average

DWI,colorFAandMDmapsfromthemagnitude-valuedU-Netreconstruction.

SupportingInformationFigS17.Fully-sampleddiffusionacquisitionwithMUSSELSreconstructiondoesnot

exhibit visible Nyquist ghost artifacts. Ghost-correction was performed using 1-dimensional navigators

acquiredforeachsliceandeachshotindividually.

SupportingInformationFigS18.Fully-sampledSAGEacquisitionwithMUSSELSreconstructionandmulti-

shot FLEET calibration data do not exhibit ghost artifacts. Ghost-correction was performed using 1-

29

dimensionalnavigatorsacquired foreach sliceandeach shot individually. 3-times scaled-up imagesare

includedtohelptheassessmentofghostlevel.