Upload
dimaia
View
35
Download
4
Embed Size (px)
DESCRIPTION
Software Integration Status and Plans . Gábor Tóth UM CRASH Team, Ann Arbor, MI October 29 , 2010. This talk is a status update and part of our response to the review team recommendations in the V&V area. Outline of Talk Software architecture and modeling schema - PowerPoint PPT Presentation
Citation preview
Software Integration Status and Plans
Gábor Tóth UM CRASH Team, Ann Arbor, MI
October 29, 2010
Page 2
This talk is a status update and part of our response to the review team recommendations in the V&V area
• Outline of Talk– Software architecture and modeling schema– New algorithms and work in progress– Testing strategy– Example tests– Version Control– Documentation
UQ integration discussed by Paul Drake and James Holloway
Page 3
CRASH Initialization
Software Architecture and Current Algorithms
HYADESMulti-material hydro with EOSMulti-group flux limited diffusionElectron heat conduction Laser heating1D or 2D LagrangianMore flexible grid options
CRASHBATSRUS
5-material hydro with EOSFlux-limited gray/multi-group diffusionFlux limited electron heat conduction2D AMR (RZ) or 3D AMR (XYZ)Split semi-implicit time steppingBlurred synthetic radiographLaser package, external EOS tables,Improve parallel I/O. Adjoint method.
PDTMulti-group discrete ordinates3D or 2D RZ adaptive gridImplicit time stepping, TBDF2Krylov methods for absorptionNearly optimal sweep schedulingMultigroup diffusion, improved I/O
Parallel comm.:(rho,u,p,Te,m)(x,y,z)
Parallel comm.:(Er,Sre,Srm)(x,y,z)
Physics informed emulator
Data reduction
Flat file(s): (,u,p,Te,Tr,Er,g,m)(x,r)
Notation blue color: work in progressred color: added this year
Postprocessorstatistical analysis
UQ
Page 4
New Algorithms
• Full energy conservation for electron/radiation physics– Total energy update but apply slope limiter on primitive variables
• Electron flux limiter– limit Spitzer-Harm flux by fraction of free-streaming heat flux
• 5-material EOS and opacity calculations– Xe, Be, Au, acrylic, polyimide– Use levelset with signed distance
• Block Adaptive Tree Library (BATL)– Efficient dynamic AMR in 1, 2 and 3D
• Split semi-implicit scheme– Solve each energy group and electrons independently.– Requires less memory. Allows PCG. Sometimes faster.
• Synthetic radiographs with blurring– Add Poisson noise due to finite photon count.– Smooth at the scale that corresponds to the pinhole size.
Page 5
Work in Progress• Laser package
– Alternative to HYADES. Full control.– Allows 3D initialization.
• External EOS/Opacity tables– Alternative to the inline EOS/opacity calculations– Collaboration with Marcel Klapisch and Michel Busquet
• Multi-level preconditioning– Using HYPRE library– Should scale better than BILU preconditioner.
• Adjoint method– Reverse execution of CRASH simulation with a series of
restart files. Provides sensitivity information.– Implemented but needs to be tested.
• Improved parallel I/O options– HDF5, striped I/O access. Prototype implemented.– Allows visualization with various software packages.– Potentially faster output from code.
• More robust feature recognition in radiographs– Based on thresholds and integral quantities
Page 6
CRASH Adjoint Solver
• Purpose of the adjoint– Accelerate UQ through efficient calculation of the sensitivity of
an output to a large number of inputs and parameters
Page 8
Verification and Testing
• New program units are implemented with unit tests– Nightly execution of many unit tests
• New features are implemented together with verification tests– Daily (18-hour) verification & full system tests are run on a 16-core Mac.– Tests should cover all aspects of the new feature including restart.– Using grid convergence studies and model-model comparison.
• Compatibility and reproducibility of features are checked with the functionality test suite– Nightly runs ensure that bugs are discovered early– Running on 8 different platforms/compilers on 1 to 4 cores: tests portability
• Parallel Scaling Tests– Weekly scaling test on 128 and 256 cores of hera.– Many-core runs require DAT. Reveals software and hardware issues.– Confirms that results are independent of the number of cores.
Page 9
Nightly unit and functionality tests:96 SWMF tests65 BATSRUS tests 4 unique to CRASH including restart
Page 10
29 verification tests: pass if converge at
the correct rate15 full system tests: pass if results have
not changed includes restart
Daily CRASH Verification and Full System Tests
Page 11
Verification Test: electron physics
• Initial condition is a semi-analytic radiation solution of R. Lowrie• Use electron EOS that mimics the radiation energy: Ee=aTe
4
• Energy exchange and heat conduction follow Kramer’s formula: ρTe-3.5
• Solution is a supercritical Mach 5 shock with a thermal front advected on a rotated AMR grid
Page 12
Verification Test: 1D electron physics
Thermal front
Hydro shockrelaxation
Page 13
Full system test:5 materials and 30 groups with electron physics on R-Z grid.
• 6 micron effective resolution with 1 AMR
• Unsplit vs. split semi-implicit schemes: 2017s vs. 1167s wall-clock time
Blurred radiograph
Page 14
Full System Test: 3D gray with elliptical nozzle Y = 0 cut Synthetic radiograph from Y
ZY
Synthetic radiograph from Z Z = 0 cut
Page 15
Strong Parallel Scaling of CRASH 2.1
3D semi-implicit multigroup-diffusion and heat conduction.2 levels of dynamic AMR, 2.6 million cells in 4x4x4 blocks.
Hera
Pleiades
Page 16
Weak Parallel Scaling of PDT on hera
• Per core: 2048 cells with 8 spatial unknowns, 80 directions and 10 energy groups.
• 1 to 12288 cores (maximum of 1.6 1011 unknowns)
Page 17
Weak Parallel Scaling of PDT on hera
Page 18
Version Release
• Development is done in a single CVS branch (HEAD version)• Code version number
– Incremented when code behavior is significantly modified– Saved with runs
• Code is tagged in CVS before and after major changes– Allows recovery of previous version– Allows comparison of results, performance, portability etc.
• Release: stable versions are created as CVS tags or branches– Released versions 1.0, 1.1, 2.0 and 2.1 so far.
• Includes BATSRUS, CRASHTEST and CRASH_data repositories.
– The released stable versions are used by UQ.
Page 19
Several Layers of Documentation
• CRASH and UQ primers• Documentation of new algorithms
– Developed during preliminary discussions– added to CVS when code is committed
• Software Development Standards document– Quality requirements, testing procedures, data naming etc.
• Documentation of changes in the CVS logs– Required for every change, generates an email
• Documentation in the source code• XML description of input parameter produces most of the user manual• User manual describes full usage including examples• CRASH full system tests serve as working examples
Page 20
Concluding Remarks
• We have made substantial progress with our algorithms• We use good software engineering practices
– Unit tests are developed with new code– Nightly functionality tests– Daily verification test suite covering all aspects of CRASH physics– All CVS changes generate notification email to developers
• Future algorithmic development– Driven by efficiency and functionality requirements– Driven by UQ
• Interaction with UQ software– Through scripts