Hittingthegroundrunning:ComputationalphysicseducationtopreparestudentsforcomputationalphysicsresearchAmyLisaGravesSwarthmoreCollegeDept.ofPhysicsandAstronomySwarthmore,PA19081AdamLightColoradoCollegeDept.ofPhysicsColoradoSprings,CO80903Keywords:ComputationalPhysics,UndergraduateResearch,PhysicsCurriculum,ResearchSkillsAbstract:Momentumexistsinthephysicscommunityforintegratingcomputationintotheundergraduatecurriculum.Oneofmanybenefitswouldbepreparationforcomputationalresearch.Ourinvestigationposesthequestionofwhichcomputationalskillsmightbebestlearnedinthecurriculum(priortoresearch)versusduringresearch.Basedonasurveyofcomputationalphysicists,wepresentevidencethatmanyrelevantskillsaredevelopednaturallyinaresearchcontextwhileothersstandoutasbestlearnedinadvance.1.Introduction Thebenefitsofcutting-edgeresearchwithundergraduatesareundeniable,incomputationalphysics(CP)oranyotherdiscipline.Researchbringsmanybenefitstostudents,withspecialonesattachedtostudentswhoareearlyintheircareer,womenand/ormembersofunderrepresentedminoritygroups[1].ThepurposeofthispaperistoaskhowtobestprepareastudentforCPresearch(CPR)whichintheU.S.typicallyhappensduringthesummer,whetheronthehomecampus,orelsewhereviaaprogramliketheNationalScienceFoundationResearchExperienceforUndergraduates(NSFREU).ArrivingpreparedataCPRlab,whetherathomeoraway,impliesagreaterchanceofthriving:comfortwiththework,higherself-conceptasaresearcher,fasterprogress,andmorelikelihoodofeventualco-authorshiponconferenceposters,talksandrefereedpapers.Whetherinalab,orasaculminatingexperienceinanundergraduatecourseorcapstone,researchis"increasinglyimportantforourstudents'careers"(Editors'Introduction,Ref.[2]). Weaskaboutpreparationasadvocatesnotjustforstudents,butfacultymembersaswell.Facultyatprincipallyundergraduateinstitutions(PUIs)typicallyhaveonlyundergraduatesastheirjuniorpartnersinresearch.Atrackrecordofcutting-edgeresearchisessentialforthecareersofmostPUIfacultymembers,makingitimperativethatthereisauthenticresearchprogressinan8-to10-weeksummer.Thecurrentpaperoriginatesina2019talk(foundatmeetings.aps.org/Meeting/MAR19/Session/F22.3).Toourknowledge,weareamongthefirst

topublicallyframethequestionofhowundergraduatestudentscan"hitthegroundrunning"inCPR. In2011,theAAPTrecommendedthatsincecomputationisintegraltothemodernpracticeofphysics,"everyphysicsandastronomydepartmentprovideitsmajorsandpotentialmajorswithappropriateinstructionincomputationalphysics".In2016theAAPTUndergraduateCurriculumTaskForce(UCTF),producedaninformativerationaleforfundamentallearningofcomputer-basedtools,andmodesofcomputationalthinkinganddoingscience[3,4,5].Thisreportidentifieschallengesdepartmentswillfaceinimplementingthesuggestions,andresourcesforfaculty.Skillsarebrokenintothreecategoriesthatbuildupononeanother:fundamental,technical,andcomputationalphysics-related.Inourstudy(seeSection3),weattempttosegregatebasic,fundamental,andmoreadvancedskillsaswellasworkplaceskillslikefindinginformation,collaboratingwithothers,andprojectmanagement(alsokeyinagraduateschoolsetting)[3]. Inadditiontodesigningnewcoursesandrepurposingexistingones,expertsrecommendleveraging"opportunitiesforcomputational...researchprojects"[5]andfosteringopportunitieswhere"undergraduateresearchisvaluedandsupported"[4].Inshort,CPskillsarerecognizedasthefruitofundergraduateCPR.Thepremiseofthispaperisthatthereverseisalsotrue.Computationalskillsareneededtoengageincomputationalresearch,whichbuildsfurtherskillsandadvancesresearchprogress.

2.ComputationintheundergraduatecurriculumWemakeadistinctionbetweencomputationalphysicseducation(teachingstudentsmorephysics,moreeffectivelywithcomputersasavehicle)andeducationinthedisciplineofcomputationalphysics[2,6,7].(UtilizingsimulationssuchasPhyslets,orPhETsimulationsfromU.ofColorado,thoughexcellentpedagogy,doesn'tcountasthelatter.)Further,thereisthewell-recognizedsplitbetweena"scientificcomputingapproach"vs.a"computationalscience"approachtoCPeducation(N.ChonackyandD.Winch'sarticle,Ref.[2]).Isitbettertoteachthetheoryofanthordersymplecticintegrator;oraskstudentstofindoneinarepositoryontheWeb,downloadandutilizeit;understandingitslimitationsandinterpretingtheresultsviathetheoryofdynamicalsystems?The"understandeverythingfromthegroundup"physicists'mindsetpredisposestowardtheformer.YetthelattermaybebetterpreparationforCPRindynamicalsystemstheory,and/ortheworldofemploymentbeyondcollege. AdecadeagotherewereonlyfiveU.S.undergraduatedegreeprogramsinCP[R.H.LandauinRef.(2)].Anupdatedreport(PICUPandtheAIPStatisticalResearchCenter)surveyingallU.S.physicsdepartmentsisforthcoming,butaroughguesscanbemadethatthisnumberhasincreased,butbylessthananorderofmagnitude.Somewherebetween25%and50%ofdepartmentsteachcomputationattheintroductoryoradvancedlevel[8,9].Thestate-of-theartroughlyadecadeagowasablyreviewedinspecialissuesofCISE(Volume8,2006)andtheAmericanJournalofPhysics[2].Valuablecurricularresourcesinthelatterincludeapedagogicalresourceletter(R.H.Landau)andaguidelinespaperbasedontwoyearsofmeetingsandsurveyswithexpertsandstakeholders(N.ChonackyandD.Winch).Thatera

mightbeconstruedasthe"2ndgeneration"ofCPeducationbooksandcourses[8],inwhichpurenumericalanalysistextsgavewaytobeingmoreorientedtowardphysics,withsupportforprojects.AcluethatCPcoursesarealreadystructuredtobenefitCPRisthatthefacultyresponsibleforCPcurricularinnovationhave,thenandnow,beenfoundtobethoseengagedinCPR[8,9]. Difficultiesininitiating/sustainingaCPcurriculumhavebeenrecognized.These"systemicforces"include[5,10]

• it'stimeconsuming(instructorsmustprepare;timefortraditionaltopicsislost)• itdisruptsthestatusquoofcoursesoffered• itrequiresnewbackground(forinstructorandstudent)• itrequiresmorethanonefacultymember-acommunity• welackresearch,fromthePERcommunityorelsewhere,oneffectiveteachingofCP

Hopefully,potentialinstructorsarereassuredthattheyarenotaloneinencounteringthesedifficulties.TheyarediscussedinlightofarecentPICUP-sponsoredfacultyworkshopinRef.[10].Onthebrighterside,therearemanyresourcesavailableforcurricularsupportandchange,including:

• campusITSdepartments,anumberofwhomarealreadyofferingtrainingsinGit,commandlinetools,theXSEDEHPCframework,andmore.

• softwarebootcamps,inpersonoronlinethroughorganizationslikesoftware-carpentry.org,datacamp.com,andLynda.com

• comPADRE.org,whichisageneralAAPT-supportedmaterialsrepositoryandincludesresourcesspecifictoCPlikeOpenSourcePhysics(OSP),andthePICUPproject.PICUPinparticulariswellalignedwiththegoalsexpressedhereandfacilitatesthe“organic”useofcomputationintraditionalcourses.

• shodor.org,whichfeaturestheJournalofComputationalScienceEducationandhoststheComputationalScienceEducationReferenceDesk(CSERD)andtheNationalComputationalScienceInstitute(NCSI).Theseresourcesprovidecurriculumandworkshopsforfacultyseekingtoteachcomputationalscience.

VisavispreparationforCPR,physicsmightgaininsightfrommaterialsscienceandengineering(MSE).ComputationalMSEisnowanexpecteddepartmentalresearchthrust.In2005mostofthetop20U.S.MSEdepartmentswereplanningtointroducecomputationalundergraduatecoursesand43%alreadyhad,by2009themajorityofdepartmentshaddonethis[11].Courseslikethesetendtoteachresearch-calibertools;suchasindustry-standardcodesfordensityfunctionaltheory,moleculardynamics,finiteelementmodelling,andthermodynamicphasediagrams;andanotebookenvironmentforcalculation/visualization[12].ThisisconsistentwiththefactthatmostMSEfacultiesprioritizecourseswithacomputationalscienceapproach;forexample,onlyasmallminorityofMSEfacultywantedstudentstolearntowritecodeandcreatenewtools[11].StatedaimsofcomputationalMSEcoursesare,asinphysics,deepenedunderstandingoftraditionaltopics.However,itisclearthatMSEcourseswhichteachundergraduatestobecomeconfidentmodelerswithstate-of-the-artsoftware

satisfytheneedtohitthegroundrunninginundergraduateresearch,graduateschoolandtheworkplace. ThePERliteraturerevealshowactivelearningprecepts[13]translatereadilytothetypicalhands-on,team-basedteachingofCPinwhichallstagesofmanipulatingcode,dataandresultsaskstudentstoconstructtheirownknowledgeofphysics.Since"Mostofourknowledgeofhowtoteach...CPisanecdotal."[5]and,asmentionedabove,fewPERstudieshaveconcernedCPeducation[8,14],thereexistatpresent"significantopportunityforthePERcommunitytosupportthedevelopmentofmaterials,teachingpracticesandassessmentstrategies."[9].AmotivationforthecurrentpaperistoexploretheopportunityforeducationalexpertsandCPRstakeholderstofactorinCPRreadiness.Wewonderwhichcomputationalskillsarebestacquiredoutsideofandbroughttothelabversuslearned(andenhanced)inthelab,andwhatpracticesmakeforinclusiveCPeducation.3.Thesurvey:Whatskillsareneededatwhatstage? RespondentswhodoCPresearchwithstudents(seeSection4)wereasked"Pleaseassessthetypeofexperienceundergraduatestudentsneedbeforeorduringtheirtimewithyouinordertobeeffective,productiveresearchersinyourcomputationalphysicslab."CategoriesAandHbelowaskedfortextanswers.ForB-G,respondentswereaskedtocheckoneormoreoffourboxes:

§ Ideally,studentshavepriorexperiencewiththese§ Studentsgetfocusedtrainingonthese§ Studentslearntheseinnaturalcourseofresearchwithme§ Thisskillisunnecessaryformywork

Surveycategorieswereasfollows:A.FieldofPhysicsinwhichIdocomputation-basedresearchB.Algorithmicthinkingskills

1. Convertingaproblemtoastep-by-stepprocedureamenabletocodingC.Basiccodingskillsandknowledge

1. Programminginsomelanguageat"hello,world!"level2. Calculation/visualizationenvironment(e.g.Mathematica,Jupyternotebook,Matlab)3. Numericalarithmetic(e.g.machineprecision,datatypes)4. Flowcontrol(loops,conditionals,etc.)5. Scopingrulesparticulartoaprogramminglanguage6. Selectingandmanipulatingdatastructures

D.Softwareengineering1. Whiteboarding2. Pseudocoding&diagramming

3. Softwarelifecyclebestpractices(design-code-implement-validate-utilize-archive)(Note:Thisanswercouldbe"no",despitea"yes"tothestepoftestingcode.)

4. KeepingaTODOlist5. Documentingcodeandkeepingdocumentationcurrent6. camelTypeandotherstrongtyping7. Interfacinghomegrowncodewithexternallibraries8. Objectorienteddesignprinciples9. Fluencyinanobjectorientedlanguage10. Parsing/understandinglegacyorinheritedcode11. Extendingfunctionalityofexistingcode12. Versioncontrol13. Creatingmodular,well-structuredcodeeasilypassedontootherstudents14. UsinganIDE

E.Scientificprojectanddatamanagement

1. Keepingalabnotebookorelectroniclog2. Insitudocumentation(READMEfiles,etc.)3. Recordinginputparameterswhencodesareruninproductionmode4. Recordingversionofcodeusedtoproduceanysetofdata5. Namingconventionsofoutputfiles6. Backingupandknowinghowtorestoredata/codes7. Sharingcodeeffectivelywithothers8. Proactivecommunicationofresultsandissues9. Collaborativemindset

F.Fundamentalscientificcomputingskills1. Runninganexecutablecodeoneisgiven2. Askingpeopleforhelpwithrunningorwritingcode3. Readingcodedocumentation4. Googlingtoanswercodingquestions5. Locatingexistingcodeandlibrariesonweb-basedservers6. Terminal/unixshellprogramming7. Unixshellscripting8. Accessingandutilizingremoteplatforms(ssh,scp,etc.)9. Submittingjobstoaqueue10. Usingaterminal-basedtexteditor11. Compilingandmaking12. Readingformatteddata13. Writingformatteddata14. Understandingcompile-timeandruntimeerrors15. Visualizinglinedata(plots)16. Visualizingimagedata(2D)17. Visualizingvolumetricdata(3D)18. Creatinganimations

G.Moreadvancedcomputationalskills

1. Highperformancecomputing2. Profilingcodeandoptimizingexecution3. Specializedhardware(GPUs,etc.)4. Usingadebuggingenvironment

H.Whathaveweforgottentoaskabout?Whatdoyourstudentsneedto"hitthegroundrunning"inyourcomputationalresearch?4.Forty-sixcomputationalphysicistsrespondtosurvey ThesurveyofSection3wassenttoatotalof105physicistsandastronomyfaculty,ofwhich35and70taughtatcolleges(PUIs)anduniversitiesrespectively.Thiscorrespondedto27and29uniqueinstitutions.WebeganwiththeU.S.NewsandWorldReport2019listoftwentytopU.S.collegesanduniversities.WebsitesfortheirPhysicsDepartmentswerescrutinizedforcomputationalresearchthatinvolvedundergraduates.Identifiedfacultymembersreceivedemailsolicitationsandalinktothesurvey.Inaddition,wecontacted19facultymembersknowntousforCPRwithundergraduates.Theoverallresponseratewas44%(substantiallyhigherforcollegesthanuniversities.)Thefieldsofrespondentswerediverse:atomicphysics,astronomy,biophysics,condensedmatter,cosmology,fluids,gravitation,materialsphysics,nonlineardynamics,nuclear,plasma,softmatter,statisticalphysics,andquantum:fieldtheory,informationtheory,mechanicsandoptics.CompellingquestionsaboutdifferencesbetweenPUIsanduniversitiesorbetweendifferentsubfieldsmustbeleftforafuturestudy. Acomprehensivelookatthesurveydataindicatesthatsomeskillsarebettersuitedtolearn"onthejob"thanothers.ThescatterplotinFigure1capturesfourdimensionsofthedata.Eachcirclecorrespondstoalabeledskill(seeSection3).Weplotskillsforwhichpriorknowledgeisdesiredvs.thoseacquirednaturallyorviafocusedtraininginthelab.Thecircleradiusindicatesfractionofrespondentsforwhichtheskillisneeded.Colorshowsfractionwhoasserttheskillisacquirednaturallyduringresearch.

Figure1:Summaryofcomputationalskillsurveydata.Eachcirclecorrespondstooneskill/surveyquestion(seeSection3).Circleradiusisscaledbythefractionofrespondentswhosestudentsneededthatskill;thelargestcircles(e.g.C4)representskillsneededby100%ofrespondents Acoarseinterpretationofthisplotisthattheskillsintheupperleftareperhapsnecessarytoteachinthecurriculum.Skillsonthelowerrightcan(perhapsshould)belearnedasneededduringthecourseoftheresearch.Theneedforbasicprogrammingskills[C1]tobedevelopedaheadoftimeisparticularlyclearinthisrepresentation.98%ofrespondentsfounditnecessary,mostagreedthatstudentsshouldhavepriorexperiencebeforebeginningresearch,andfewbelieveditdevelopsnaturallyduringthecourseofresearch.Ofthe52skillssurveyed,sixofthemhadallrespondentsidentifythemasnecessary:

• [B1]Convertingaproblemtoastep-by-stepprocedureamenabletocoding• [C4]Flowcontrol(loops,conditionals,etc.)• [E8]Proactivecommunicationofresultsandissues• [F2]Askingpeopleforhelpwithrunningorwritingcode• [F4]Googlingtoanswercodingquestions• [F15]Visualizinglinedata(plots)• [F16]Visualizingimagedata(2D)

Thetypeofexperiencerespondentsidentifiedasnecessaryforstudentstohavewiththeseskillswasnotuniform,however.Responsesfor[B1]and[E8]above,aswellas[C1](basicprogramming)and[E4](codeversion/outputassociation)arecomparedinthepiechartsofFigure2.Thevastmajorityofrespondentsvaluedtheseskills,andtheycoverarangeofthetradeoffbetweenpriorknowledgeandnaturalskilldevelopment.

C1

F1

C4

D6

G3D14

C2B1F2

F3

F15D5E9 C3F5

D4F10

F7

D9

D8D3

D1

C6F11

F6C5F13

F8E1

F4

F12F17

F18

G2D2F9D12 E2G4

G1D7

E7F14

F16E8

E6D10 E5

E4

D13E3,D11

Anoverwhelmingnumber(80%)ofrespondentsemphasizedaneedforpriorexperiencewithprogramming[C1].Respondentssplitfairlyevenlyonwhethertheskill[B1]oftranslatingaproblemintoproceduralcodewasabestlearnedaheadoftimeornaturallyinthecourseofresearch,withanother15%advocatingfocusedtraining.Athirdofrespondentsindicatedthatstudentsshouldcomeinwith[E8],abilitytoproactivelycommunicateresultsandissues.Themundanebutvitalskill,valuedby93%ofrespondents,ofmatchingcodeversionstodataproduced[E4]waslearnedprimarilyonthejob.

Figure2:Breakdownofresponsesforparticularskillsdiscussedintext.Primaryresponses:blue,redandyellow;respondentschoosingtwoprimaryresponses:orange,purpleandgreen;choosingallthree:white;categorizingskillasunnecessary:gray.5.DosurveyresultsmaptotypicalCPcurricula? Itisarebuttablepresumptionthataphysicscourseincomputationwillinstillmany,butnotall,skillsideallyneededpriortoCPR.Forexample,considerthefollowingfourexcellentundergraduatecourses:•Sharma'sfreshman-levelcourseforpotentialSTEMmajorsatWagnerCollege[15]exposesstudentstoworkflowsinproducing,analyzingandvisualizingtechnicaldata.Withnocomputerscienceprerequisites,itdistancesitselffrombeingaprogrammingcourse.Mathematica,open-sourcemoleculardynamics,andmolecularvisualizationpackagesaretaught.AswithMSEcomputingcoursesdiscussedinSection2,theemphasisisonusinghighlevelplatformsforinsightintohighlevelquestions.IntegraltothecoursearemanipulatingandvisualizingdatastructuresinMathematica,frommolecularmechanicssimulation,andfromProteinDataBankfiles.Someofthesubstantialfinalprojectshavealevelofinnovativeexplorationwhichqualifiesascomputationalresearch.

B1

E8 E4

C1

natural

trainingprior

prior+training

training+natural

natural+prior

none

all

Colorkey

•CaballeroandPollock'sPER-informed,Mathematica-based,intermediateclassicalmechanicscourseatU.ColoradoatBoulderseeksbothtodeepenanunderstandingoftraditionaltopicsandtopreparestudentsforprofessionalphysicsresearch[14].Itispragmaticintermsofstudentandfacultytime.Forexample,theoryoferroranalysisisreplacedwithcommonsensecheckslikerunninglimitingcases,a"criticalpartofthemodelingcycle"(Editors'Introduction,Ref.[2])..•BrynMawrCollegehasdevelopedanextensivesetofmaterialsforbothinstructorsandstudents[16].FundedbyTIDES,aDiversity/EquityinitiativeoftheAACUanditsProjectKaleidoscope(PKAL),ateamdeveloped14iPython/Jupyternotebook-basedmoduleswhichembedcomputationallessonsinphysics/engineeringandsocialcontext.Students'electronicportfolioscontaincomputingworkplusprofilesofcomputationalscientistsandreflectionsonhowwhatwaslearnedintersectswithpersonalgoals.Itisclear,forexamplefromtheNationalCenterforScienceandEngineeringStatisticsWomen,Minorities,andPersonswithDisabilitiesinScienceandEngineering:2019.SpecialReportNSF19-304,thatthephysicalandcomputersciencesarefieldsthatpersistentlyskewedawayfromwomenandpeopleofcolor.Thus,goalsofequityandinclusionaresingularlyvaluable. •SwarthmoreCollege'scomputationallabhasundergonemanyincarnationsin30years-usingBasic,IDL,Matlab,Mathematica,and(currently)withiPython/Jupyternotebooks.Currently,thecourseis7weekslongandprecededbyarecommendedworkshop,runbytheSoftwareCarpentryFoundationofSanFrancisco,CAandcoveringterminalediting,Unixshell,anduseofalocalGithubforversioncontrol.Topicsinnumericalanalysisareintegratedwithphysicalapplications,withafinal,syntheticexerciseonLIGOgravitationalwavedata. InTableIwe'venotedwhichsurveyedskillsarelistedasexplicitcoursegoals;withthecaveatthatnamesofcategoriesB-Garenotrigorousidentifiers,andareopentointerpretation. Wagner Colorado BrynMawr SwarthmoreB.Algorithmicthinking

1 1 1 1

C.Basiccoding

1-6 2,3 1-4,6 1,2,4-6

D.Softwareengineering

6,11 None 2,5,7,8,11,13,14 5,7,10,11,13

E.ScientificProgramming

1,5-9 1,8,9 1,9 None

F.Fundamentalcomputing

1-4,12,13,15-18 1,2,4,15-18 1-6,10,12-16 1-4,12-16

G.Moreadvanced

None None 2,4 None

TableI.SkillsdetailedinSection3whichwerealsocoursegoals.

Notes:i.AbsenceofD3isnotindicativeofalackof"validation",sinceseveralcoursesincludedtestingofknowncases.ii.D14mayrefertotheJupyterenvironment,notaseparateIDE. Weinvitereaderstocross-compareTableIwithCPRexpertprioritiesdisplayedinFigures1and2.Alsoconsiderthehandfulofskillswhichnoneofthefourcourseshadasaninstructionalgoal:D1,D3,D4,D9,D12;E2-4;F7-9,F11;G1andG3.TheseomissionsarequiteconsistentwithFigure1.Theleastconsistentwas[D4]keepingaTODOlist;evenso,lessthan30%ofrespondentswantedpriorexperience,around50%saiditwouldbelearnedonthejob,and20%itwasnotneededatall.Understandably,someitemsareabsentbecausetheyareoutsidethetechnicalscopeofcoursetools(e.g.terminaleditingusingahighlevellanguage,HPC).Evenso,studentsdidfeelthelackofsomeofthese,suchastheabilitytodebugtheirprograms. Outsidethetechnicalreachofmanycoursearecertainskillsinwhichroughly20%ofsurveyrespondentswishedpriortraining:[F7]Unixshellscripting,[F8]connectingwithremoteplatforms,[F11]compilingandmaking,and[E2-4]writingREADMEsandkeepingtrackofversionsandinputparameters.(Item[F6]terminalandUnixbasics,wasequallydesirousforCPR,andagoalofonlyoneofthefourcourses.)Ontheotherhand,skill[D4]KeepingaTODOlist,wasoneinwhichroughly30%ofrespondentswantedpriorpreparation.Thispre-professionalskillworthteachingwouldnotstretchthetechnicalscopeofanycourse.6.AdviceandinnovationfromfolksdoingCPR OneCPcoursewhichdeservescarefulmention[17]isfocusedonCPRreadinessandrootedinexpertadvice.BurkeandAtherton,fortheir2015TuftsUniversitycourse(mostlyundergraduatesatroughlyasophomorelevel)modifiedtheirinitiallistofcompetenciesbasedoninterviewswithfacultyandpostdocsactiveinCPR.Theirevidence-basedmodelofexpertproblemsolvingwaslesslinearthantheiroriginalmodel(Figure3below),andincludedmeta-competenciesseldomfoundintraditionalundergraduateCPcourses.

Figure3:Adapted from Figures 1 and 2 of Am. J. Phys. 85 , 301 (2017), with the permission of the American Association of Physics Teachers. Thelearninggoals,inanutshell,were:(1)modelaphysicalsystem(2)design-implement-validatecode(3)understandtheimportanceofnumericalanalysis(4)testhypothesesviavisualizationandothermeans(5)connecttheory,experimentandsimulation.Aphysics-content-driven,project-basedcourseisatime-honoredwaytoprogressivelyintroducesuccessivelydeeperknowledgeofthevariouselementsofCP,afactinevidenceinmanywidely-usedCPtextbooks.(SeeforexampleRebbi'sarticleinRef.[2];Gould,TobachnikandChristian'sIntroductiontoComputerSimulationMethods,2006;orLandau,Päez,andBordeianu'sComputationalPhysics:ProblemSolvingwithPython,2015).Ithasalsobeennotedthatsuchcoursesarebothconsistentwiththeethosofaliberalartseducationandvaluablepreparationforsizeableresearchprojects[18],towitCPR,inthefuture.InRef.[17],learningwasimplementedviashortprojectsinclassical,quantumandstatisticalphysics,culminatinginafinalprojectofthestudent'schoice.Asubsetoflearningobjectiveswerehighlightedforeachproject;theseincludedauthenticresearch-relatedactivitieslikereadingpapers,consultingInternetresources,workingingroups,runningcode,testingperformance,documenting,andversioncontrol.MathematicaandPythonwerechosenastherequiredprogramminglanguages,butstudentswereallowedtouseothersforthefinalproject,providingyetanotherpragmaticlesson:cross-comparingdifferentlanguagesincontext.Courseevaluationsindicatedsuccess,butpointedtoconstructivechangessuchasadditionalscaffoldedlearningforstudentswithdifferentbackgrounds,andbettermanagementofgroupdynamics.

initialmodel evidence-basedmodel

includesmeta-practiceslikeexternaldocumentationandversioncontrol

Turningtoournewsurveydata,suggestionsofferedinItemH(seeSection3)canbedistilledintovariouscategories.SomerespondentsI.assertedthatstudentsshould

• haveprogrammingandprerequisitemathematics• beabletoconvertaproblemtoastepwiseprocedureforcoding(surveyItemB1)• knowspecificnumericaltechniques,e.g.FFT,nonlinearcurvefitting,orlinearalgebra• beabletorefactoranexistingcode• beabletocreateneworsubstantiallyextendalgorithms• knowtotestacode-evenifitisviaa"realitycheck"viaaknowncase.(Some

respondentsworkinenvironmentswhichdon'thavedebuggingcapabilities.)• thinkincommon-sensetermsaboutalgorithmsandworkflow.Forexample

"tostructureandautomatecomputationssothattheoutputofonecalculationcanbeefficientlypassedasinputtothenext"

• tohavealgorithmicthinkingskillspairedwithanassociatedunderstandingoftheunderlyingscience

• beproactivelycommunicative• beinterested,dedicated,andmotivated

II.wonderedhowtoteachtheirstudents

• tobe"fearless"abouttryingwhatmaynotwork,andpersistuntiltheysucceed• "softskills"ofcomputinglikecommenting/versioncontrol/writingREADMEs• skillsthattheythemselvesmaynothavelearned,ordon'temploytotheextentthat

best-practiceswoulddictate.Asonesaid:"Mycodingexperiencewasmostlyself-taughtandIgraduallylearnedthe(se)lessonsthehardway".

Severalrespondentsdescribedaself-pacedbootcampexperiencewhichtheyhavedevisedforstudents.Forexample:"Ihavethemlogintomycomputationalclustertoworkthroughsomestandardcannedexercises...studentsworkinteams.Twostudentscanbeworkingonslightlydifferentversionsofthesameproblem...Thatway,theyhaveacommunitytotalkto,buttheyhaveownershipovertheirownresults."Finally,whileseveralrespondentsstruggledwiththerarityofhavingstudentswhohitthegroundrunning;atleastone(fromanR1university)wasOKwiththis,expectingundergraduatestoarriveneedinginstructioninverybasicskills.

AfinalexpertperspectivecomesfromCaballero'scodednotesfrominterviewswithCPRprofessionalsinacademiaandindustry.Ref.[19]breaksskillsintoseveralusefulcategories,mostofwhichappearedonoursurveydirectlyorwereaddressedinoneormorefreeresponses.Theonlycategorywithoutrepresentationwasmeta-knowledgeofwhata

computationalsolutioncandeliver.ResonancesbetweentheskillsinRef.[19]andoursurveydataare:

• Knowledgeofcertaincomputationalmethods,bothnumericalanddata-centered(freeresponses)

• Knowledgeofspecificphysicalconcepts(freeresponses)• Movefluidlybetweenmath,physicsandcomputation-relatedideas(free

responses)• Professionalpracticeslikeplanning/writingpseudocodeanddebugging

(addresseddirectly)• Dataanalysisandvisualization(addresseddirectly)• Computationalthinking:breakingacomplextaskintostepswhichcanbe

encoded(addresseddirectly)• Meta-knowledgeofwhatacomputationalsolutioncandeliver(notaddressed)• Professionalpracticeslikedocumentingcodeandversioncontrol(addressed

directly)IntervieweesinRef.[19]alsomadeapointechoedbyseveralofoursurveyrespondents:Theywereself-taughtinprofessionalCPpractices.Therewasagreement,however,thatthiswasnotasituationthatshouldcarryforwardintotheeducationofthenextgenerationofphysicists.7.Conclusions CPissuccinctlydefinedbytheEuropeanInstituteofPhysics(IOP)websiteas"thescienceofusingcomputerstoassistinginthesolutionofphysicalproblems,andtofurtherphysicsresearch".TheIOP'sComputationalPhysicsGrouplistsaremarkablearrayofcomputationalphysicssubfields,suggestingitmaybehopelesstoidentifyasingletypeofworthwhileundergraduatepreparationforall.Nevertheless,theCPRandPERcommunitieswouldbenefitfromcomingtogethertoclassifytypesofskillsandbestteachingpractices.Thisissueisexpressedbyonesurveyrespondent,whowishestheyhadanestablishedprogramtobringstudentsuptoaneededlevelofpreparedness"insteadofitbeingacase-by-case,adhoctrainingsituation"whichworksforasmallnumbersofstudents,butnotinperpetuitywithlargenumbersofstudentsintheirresearchpipeline.Anotherrespondentremarksthatonemightsimilarlywishtoprepareincominggraduatestudentswithneededbreadthofskills.Giventheirdata-intensivefield,thispersonremarks"youunfortunatelyhavetoeitherteachthemcomputationorphysics.Astheseskillsarelearnedindifferentschoolswithinuniversities...I'mnotawareofanyundergraduateprogramthatIthinkstreamlinespeopletoworkwithme." Inconclusion,wehopethatourcurrentstudyisa"proofofprinciple"thatexpertsandstakeholderscanteachuswhichskillsareneeded,yetnoteasilymastered,duringabriefstintdoingundergraduateCPR.Inthisway,weinformourselveshowtostructureCPcurriculatohelpstudentshitthegroundrunning.

8.AcknowledgementsWearegratefultoJoanAdlerandRubinLandaufortheopportunitytopresentourwork.Wethankthecolleagueswhogenerouslyrespondedtoourlengthysurvey.AdditionalthankstoTonyAtherton,LorenaBarba,DanielCaballero,CatherineCrouch,RobertHilborn,PaulJacobs,MichaelLerner,MarkMatlin,RobertPanoff,ArjenduPattanayak,AndrewRuether,ArunkumarSharma,andChristopherTunnell.WegratefullyacknowledgefundingfromtheOfficeoftheProvostand(AG)aJamesMichenerFacultyFellowshipofSwarthmoreCollege.References1.A.Carpi,D.M.Ronan,H.M.Falconer,N.H.Lents.Cultivatingminorityscientists:Undergraduateresearchincreasesself-efficacyandcareerambitionsforunderrepresentedstudentsinSTEM.JournalofResearchinScienceTeaching54,169(2017).

2."ThemeDoubleIssue:IncorporatingComputationintotheUndergraduateCurriculum"(W.ChristianandB.Ambrose,eds.)Am.J.Phys.76,293(2008).

3.E.BehringerandL.Engelhardt,GuestEditorial:AAPTRecommendationsforcomputationalphysicsintheundergraduatephysicscurriculum,andthePartnershipforIntegratingComputationintoUndergraduatePhysics,Am.J.Phys.85,325(2017).

4.L.McNeilandP.Heron,PreparingPhysicsStudentsfor21stCenturyCareers,PhysicsToday70,38(2017).

5.AAPTUndergraduateCurriculumTaskForce,ComputationalPhysicsintheUndergraduateCurriculum:Executivesummary(2016)[PDFfile]Retrievedfromhttps://www.aapt.org/Resources/upload/AAPT_UCTF_CompPhysReport_final_B.pdf

6.R.H.Landau,ComputationalPhysics:ABetterModelforPhysicsEducation?CISE8,22(2006).

7.R.M.Panoff,PhysicsattheSpeedofRight:ChallengesinComputationalPhysicsEducation(2011)Retrievedfromhttp://www.shodor.org/talks/computationalphysics.

8.R.Martin,Undergraduatecomputationalphysicseducation:unevenhistoryandpromisingfuture,J.Phys.:Conf.Ser.759,012005(2016).

9.M.D.CaballeroandL.MernerPrevalenceandnatureofcomputationalinstructioninundergraduatephysicsprogramsacrosstheUnitedStatesPhys.Rev.Phys.Educ.Res.14,020129(2018).

10.A.Leary,P.W.IrvingandM.D.Caballero,Thedifficultiesassociatedwithintegratingcomputationintoundergraduatephysics.(PERResearchConference,Washington,DC,2018).Retrievedfromhttps://www.compadre.org/per/items/detail.cfm?ID=14812

11.K.Thornton,S.Nola,R.E.Garcia,M.AstaandG.B.OlsonComputationalMaterialsScienceandEngineeringEducation:ASurveyofTrendsandNeeds,JOM61,12 (2009).

12.R.Mansbach,A.Ferguson,K.Kilian,D.R.Krogstad,M.WestandG.L.Herman,Reforminganundergraduatematerialssciencecurriculumwithcomputationalmodules,J.MaterialsEducation38,161(2016).

13.D.E.MeltzerandR.K.Thornton,ResourceLetterALIP-1:Active-LearningInstructioninPhysics,Am.J.Phys.80,478(2012).

14.M.D.CaballeroandS.J.Pollock,AModelforIncorporatingComputationwithoutChangingtheCourse:Anexamplefrommiddle-divisionclassicalmechanics,Am.J.Phys.82,231(2014).

15.A.K.Sharma,AmodelScientificComputingcourseforfreshmanstudentsatliberalartscolleges,J.ofComp.Sci.Ed.8,2(2017).

16.M.Matlin,D.BlankandE.McCormackEnhancingComputationandInformationScienceLearningOpportunitiesforWomenLeadersinSTEM(2017)Retrievedfrom

blendedlearning.blogs.brynmawr.edu/tides-computational-skill-building/

17.C.J.BurkeandT.J.Atherton,Developingaproject-basedcomputationalphysicscoursegroundedinexpertpractice,Am.J.Phys.85,301(2017).

18.R.DominguezandB.Huff,Theroleofcomputationalphysicsintheliberalartscurriculum,J.ofPhys.ConferenceSeries640,012061(2015).

19.D.M.Caballero,Computationacrossthecurriculum:Whatskillsareneeded?(2015)[PDFfile]Retrievedfromarxiv.org/pdf/1507.00533.pdf

AuthorBiographies:AmyGraves(formerlyAmyBug)istheWalterKempProfessorintheNaturalSciencesatSwarthmoreCollege.EducatedinmathematicsandphysicsatWilliamsCollegeandinphysicsatM.I.T.,hercomputationalsoftmatterresearchhasbeensupportedbytheAmericanChemicalSociety'sPetroleumResearchFundandtheNationalScienceFoundation'sDivisionofMaterialsResearch.Shealsostudiescomputationinthephysicscurriculum,andraceandgenderissuesinSTEM.SheisaFellowoftheAmericanPhysicalSociety,amemberofitsCommitteeontheStatusofWomeninPhysics(CSWP),andontheEditorialBoardoftheAmericanJournalofPhysics.Hercontactemailisabug1@swarthmore.edu.AdamLightisanAssistantProfessorintheDepartmentofPhysicsatColoradoCollege.EducatedatCaseWesternReserveUniversityandTheUniversityofColoradoBoulder,heisinterestedinbasicandappliedplasmaphysics.Dr.LightisamemberoftheAmericanPhysicalSociety.Hiscontactemailisalight@coloradocollege.edu.