Department of Energy · A. Remedial: Macsyma is a general purpose program for symbolic and numerical mathematical manipulations that has been widely accepted and used in the scientific

Department of EnergyOffice of Civilian Radioactive Waste Management

45s, I/ - Yucca Mountain Site Charactenzation OfficeP.O. Box 98608

Las Vegas, NV 89193-8608

NOV 2 9 1994

L. Dale FoustTechnical Project Officer

for Yucca MountainSite Characterization Project

TRW Environmental Safety Systems, Inc.Bank of America Center, Suite P-110101 Convention Center DriveLas Vegas, NV 89109

EVALUATION OF AMENDED RESPONSE TO CORRECTIVE ACTION REQUEST(CAR) YM-94-076 RESULTING FROM YUCCA MOUNTAIN QUALITY ASSURANCEDIVISION (YMQAD) AUDIT YMP-94-01 OF CIVILIAN RADIOACTIVE WASTEMANAGEMENT SYSTEM MANAGEMENT AND OPERATING CONTRACTOR(SCPB: NA)

The YMQAD staff has evaluated the amended response toCAR YM-94-076. The response has been determined to besatisfactory. Verification of completion of the correctiveaction will be performed after the effective date provided.Any extension to this date must be requested in writing, withappropriate justification, prior to the date. Please send acopy of extension requests to Deborah Sult, YMQAD/QATSS, 101Convention Center Drive, Suite 640, Las Vegas, Nevada 89109.

If you have any questions, please contact either Robert B.Constable at 794-7945 or John F. Pelletier at 794-7538.

Richard E. Spence, DirectorYMQAD:RBC-991 Yucca Mountain Quality Assurance Division

Enclosure:CAR YM-94-076

9412050263 941129 J S >'YMP-5 PDR WASTE

WM-I1 PDR

L. Dale Foust -2- NOV 2 9 1994

cc wencl:NRC, Washington, DC

S. W. Zimmerman, NWPO, Carson City, NVT. A. Wood, HQ (RW-14) FORSR. L. Robertson, M&O/TRW, Vienna, VAR. P. Ruth, M&O/TRW, Las Vegas, NVRichard Jiu, M&O/Duke, Las Vegas, NV

cc w/o encl:W. L. Belke, NRC, Las Vegas, NVD. G. Sult, YMQAD/QATSS, Las Vegas, NV

W \_,*. f8P~ORtC11ALTHIS IS A RED STAMP

OFFICE OF CIVILIAN 8 CAR NO.: H-94-076PAGE: ....... OF 2.RADIOACTIVE WASTE MANAGEMENT O

U.S. DEPARTMENT OF ENERGYWASMINGTON, D.C.

CORRECTIVE ACTION REQUEST1 Controlling Document 2 Related Report No.

OCRWH QARD DOE/RW-0333P, Revision 01 YEW-94-01

8 Responsible Oanization 4 Discussed WithMO| Jim Eouseworth

6 Requirement:1) QARD DoE/RW-0333P, Spplement 1.2.5 states:

That acquired software shall be docuneted sufficiently to demonstratethe ability of the software to meet the needs of the affected organization.(Continued on next page)

6 Adverse Condition:

Contrary to the above, QoRD requirements, Waste solation Evaluation ofConstruction Water for Package 2C, DItEh3E00000-01717-2200-00008 and WasteIsolation Evaluation: racers, Fluids, and aterials for Use in thePackage 2C Esploratory Studies Facility Construction, DEMS0000-01717-2200-00007, indicate that the software package Macsyma (acsyma Inc.PC Version 417.125) were used in the evaluation of both WIEs. Specifi-cally the WE for Pracers, Fluids, and Materials uses the Macsymai programto provide the analytical solution of an integrated three dimensionaldispersive transport equation. Section 4.2 of the WIE identifies thesaturated advectlon-dispertion model for nonreactive chemical transport asa critical model for nonreactive chemical transport as a critical model.There is no docuentation that this software meets the needs of theaffected organization.

Contrary to the above, regarding HLP-19-1, Revision 0, there is noobjective evidence that requirements for verification of Macysma softwarein Step 5.1.2, were performed before use in the WIEs.

NOT:Zplementing Line Procedure NlE-19-1, Revision 0 states that MHacsyma is notconsidered scientific and (or) engineering software. There is no(Continued on next page)

Does a Significant Condition 10 Does a stop work condition exist? 3 Response Due Date:Adverse to Quality exdst? Yes__ Nqo_ Yes_ No_ ; If Yes - Attach copy of SWO 20 Working DayslfYes.CheckOne:OAOB CODCD[E tfYes,CheckOne: A 06 OC Frozm Isuance

t1 Required Actions: CD] Remedial E Extent of Deficiency FE Preclude Recurrence 0 Root Cause Determination

12 Recommended Actons:

Initiator ,K.A&fIZ4-..7 lrfiato ,/ 141ssuar.- oid. F

John F. Petier 8 i oate

15 Response Accepted 16 Response Accepted

QA 2 M^()2/. D._ Date,, QADD Date17lfTIl IFsbp, 1 8 Ameri" e epled

C L 4 IDaA Qm A D RWt Date °19 e Actions Verified --. 20 Closure Approved by:

OAR Date QADD Date

Exhlbt AP-16.1.1 ENCLOSURE RV. 0=/94

OFFICE OF CIVILIAN 8 CAR No.: YM-94-076RADIOACTIVE WASTE MANAGEMENT PAGE: 2 F 2

U.S. DEPARTMENT OF ENERGY OAWASHINGTON, D.C.

CORRECTIVE ACTION REQUEST (CONTINUATION PAGE)

5 Requirements (continued)

2) QXRD D/RW-0333P, Supplement 1.2.6 states:

That software activities shall be documented sufficiently to demonstratethe ability of the software to meet the needs of the affectedorganization.

3) Implementing Line Procedure ?lP-19-1, Revision 0, Section 5.1.2 states:

That formulas and algorithms used during the analytical process withstatistical analysis and interactive numeric and symbolic algebrasoftware, should be verified by both of the following methods:

a. Extracted directly from an appropriate and recognized sourcee.g.,textbook, handbook) and entered correctly.

b. Verifying the results of their results of their use by means of asimilar software package or an alternate calculation.

6 Adverse Condition (continued)

objective evidence that this software is not scientific and (or)engineering software and thus exempted from the QRD requirements. Thissoftware is used in WI1s.

Further, 5.1.2 specifically states that formulas and algorithms usedduring the analytical process with statistical analysis and interactivenumeric and symbolic algebra software, should be verified by both of thefollowing methods:

There is no objective evidence that these requirements were performedbefore use In the W13s.

Exbbft QAP-1 8.1.2 REV. 2/14/94

Exhbk CUUP-1 8.1.2 REV. 2AN

OFFICE OF CIVILIANRADIOACTIVE WASTE MANAGEMENT PAGE 1 OF2

U.S. DEPARTMENT OF ENERGYWASHINGTON, D.C.


1. Corrective Action Response for CAR YM-94-076

A. Remedial: Macsyma is a general purpose program for symbolic and numerical mathematical manipulations that has beenwidely accepted and used in the scientific and engineering community for more than a decade. As opposed to typicalscientific and engineering software, there is no standard set of model equations that are solved. Each use of Macs Alainvolves the creation of unique mathematical expressions that require verification for the specific application. Ir his respect,Macsyma is similar to commercial spreadsheet software like Lotus. Therefore. the standard use of Macsyma inconsistentwith global verification and validation (V&V) concepts for scientific and engineering sofware and its exemption from theserequirements is justified.

Objective evidence of caculation verification was not provided in the preparation of the identified Waste Isolation Evaluations(WIEs). Verification checks were performed as good engineering practice. but are not formally documented under ourcurrent procedures. Such evidence will be provided by incorporating the verification steps described below as an attachementto the affected WIEs:

1. Identify those calculations performed using Macsysna.

2. Comparison of symbolic mathematical expressions similar to those used in the WIEs that were derived by Macsyma withexpressions given in mathematical handbooks.

3. Present hand calculations for numerical evaluations performed iith Macsyma that are representative of the types ofcalculations performed in the affected WlEs.

B. Extent of Deficiency: The identified deficiency is present only in the following WIEs:

Waste Isolation Evaluation, Construction for Package 2C Excavation of the ESF North RampDI: BABEOOOOO-01717-2200-00008 Rev. 00

Waste Isolation Evaluation: Tracers, Fluids, and Materials for Use in the Package 2C Exploratory Studies Facility ConstructionDI: BABE00000-01717-2200-00007 Rev. 00

In addition, we recognize that a formal system within the M&O does not exist for identifying whether software is or is notexempt from QARD Supplement I requirements for scientific or engineering sofmwre. These decisions are currently made atthe group manager level and, in the absence of specific criteria, may result in a nonuniform application of QA requirementsfor software.

C. Preclude Recurrence: NLP-3-17 will be revised to incorporate verification criteria for use of commercial off-the-shelf (COTS)software not subject to global V&V requirements as described in NLP-19-1 Rev. 00. Two members of the PerformanceAssessment and Modeling Department received training on NLP-19-1 Rev. 00 on 8110194.

A formal M&O-wide system for identifying if software falls under the QARD definition of scientific or engineering softwarewill be developed.

'is t 4r L E .1d£ 3 - Mh - 11 E.....AP161. a LEV. 7 194-

Exhibit CIAPACI.2 I .,REV. 06127/94

-

OFFICE OF CIVILIANRADIOACTIVE WASTE MANAGEMENT


a.CAR NO. YM9476

PAGE X OF 2

QA


2. Response Actions;

Revise affected WIEs to incorporate objective evidence of Macsya calculation verification. Revise NLP-3-17 to incorporateverification criteria for use of COTS software not subject to global V&V requirements as described i' 4LP-19-l Rev. 00.Develop formal M&O systemiforclassification of software with respect to the applicability of QARD :;ipplement 1.

Remedial ActionResponsible: James E. HouseworthAnticipated Completion: 8/17/94

Preclude RecurrenceResponsible: Albin Brandstetter and Nathaniel HodgsonAnticipated Completion: 12131/94

3. Response Approved: Date: ______/q_

Reg o and Techmcafvaluation Manager

Ex.i . A- _ .. E.0179

Exhibit QAP-1 5.1 .2 REV. 06/27/94

N - '�W) I,,- 8.



CAR NO. XM-4 6766_

PAGE I OF 2_QA



A. Remedial:

Objective evidence of caculation verification was not provided in the preparation of the identified Waste Isolation Evaluations(WIEs). Verification checks were performed as good engineering practice, but are not formally documented under ourcrent procedures. Such evidence will be provided by incorporating the verification steps described below as an aftachementto the affected WEs:

1. Identify those calculations performed using Masysma.

2. Comparison of symbolic mathematical expressions simila to those used in the WIEs that were derived by Macsyma withexpressions given in mathematical handbooks.

3. Present hand calculations for numerical evaluations performed with Macsyma that are representative of the types ofcalculations performed in the affected WIEs.

The above actions are intended to satisfy the confidence that the two WIEs are adequate to support Design Package 2Crelease, and does not represent a solution to the longer term issue addressed in the next paragraph.

B. Extent of Deficiency: We have reviewed all WIEs and the only ones where Macsyma or similar software was used are listedhere:


Waste Isolation Evaluation: Tracers, Fluids, and Materials for Use in the Package 2C Exploratory Studies Facility ConstructionDI: BABEOOOO01717-2200-00007 Rev. 00

C. Preclude Recurrence: NLP-3-17 will be revised to incorporate the requirement for the identification of the verificationmethod(s) used on the results of the use of software programs during the performance of Waste Isolation Evaluations. Tomembers of the Performance Assessment and Modeling Department received training on NLP-19-1 Rev. 00 on 8/10/94.

Develop a process to identify the applicability of Scientific & Engineering Software controls on formulas and algorithms thatare used as input to non-S & E software, and the resulting effects. We shall address this issue in revisions of the QAP-19series procedures and NLP-19-1.

2. Response Actions

1. Revise affected WIEs to incorporate objective evidence of Macsyma calculation verification.

2. Revise QAP-19 series procedures and NLP-19-1 to address controls on formulas and algorithms used as input to non S & Esoftware.

3. Revise NLP-3-17 to incorporate requirements to identify verification method(s) for results of computer analyses used inWIEs.

Exhibit CAP-16.1.20 /- / ,,

REV. 06127/94I I I -VI I I I ^ I., /I I 'r 1� I

--

OFFICE OF CIVIUANRADIOACTIVE WASTE MANAGEMENT


8.CAR NO. YM.2.4476

PAGE 2 OF 2 _QA

- CORRECTIVE ACTION REQUEST (CONTINUATION PAGE)

Remedial Action

Revision of Package 2C WEEsResponsible: James Houseworth and David SaaAnticipated Completion: 8/19/94

Preclude Recurrence

Revision of QAP-19 series proceduresResponsible: Warren StandleyAnticipated Completion: 12131/94

Revision of NLP-19-1Responsible: Nathaniel Hodgson

Anticipated Completion: 12131/94

Revision of NLP-3-17 Rev. 01Responsible: Albin BrandstetterAnticipated Completion: 12/31/94

3. Response Approved: 9 y - vuReggatory and Technical Evaluation Manager

Date: -_ _- 9 _

Exhibit OAP*1 8.1.2 REV. 08127194Exhibt QAP1 S6.1.2 REV. 08127194

.

OFFICE OF CMLIAN CAR NO. 9476RADIOACTIVE WASTE MANAGEMENT PAGE I OF2

-U.S. DEPARTMENT OF ENERGY QAWASHINGTON, D.C.



A. Remedial:

Objective evidence of calculation verification was not provided in the preparation of the identified Waste IsolationEvaluations (WIEs). Verification checks were performed as good engineering practice, but are not formally documentedunder our current procedures. Such evidence will be provided by incorporating the verification steps described below as anattachement to the affected WIEs:

1. Identify those calculations performed using Macsysma.

2. Comparison of symbolic mathematical expressions similar to those used in the WIEs that were derived by Macsyma withexpressions given in mathematical handbooks.


The above actions are intended to satisfy the confidence that the two WIE's are adequate to support Design Package 2Crelease, and do not represent a solution to the longer term issue addressed in the section on Preclude Recurrence.



Waste Isolation Evaluation: Tracers, Fluids, and Materials for Use in the Package 2C Exploratory Studies Facility ConstructionDI: BABEOOOO-01717-2200-00007 Rev. 00

C. Preclude Recurrence: NLP-3-17 will be revised to incorporate the requirement for the identification of the verificationmethod(s) used on the results of the use of software programs during the performance of Waste Isolation Evaluations. Twomembers of the Performance Assessment and Modeling Department received training on NLP-19-1 Rev. 00 on 8/10/94.

Develop a classification process to determine if software falls under the QARD Supplement I requirements for Scientific &Engineering (S&E) software. This process will also address the applicability of S&E software controls on formulas andalgorithms that are used as input to non-S&E software, and the resulting effects. We shall address these issues in revisionsof the QAP-19 series procedures and NLP-19-1.

2. Response Actions

1. Revise affected WIEs to incorporate objective evidence of Macsyna calculation verification.

2. Revise QAP-19 series procedures and NLP-19-1 to address classification of software to determine applicability of QARDSupplement I requirements and applicablity of S&E software controls on formulas and algorithms used as input to non-S&Esoftware.

3. Revise NLP-3-17 to incorporate requirements to identify verification method(s) for results of computer analyses used inW[Es.

Exhibit GAP-1 6.1.2/ , REV. 08127194o/s A ,/. ,/ f11 , *A,/



8.CAR NO. Y269447

PAGE 2 OF2

QA


Remedial Action

Revision of Package 2C WEsResponsible: James Houseworth and David SassaniAnticipated Completion: 8119/94

Preclude Recurrence

Revision of QAP-19 series proceduresResponsible: Warren Standley

Anticipated Completion: 12/31/94

Revision of NLP-19-1Responsible: Nathaniel Hodgson

Anticipated Completion: 12/31/94

Revision of NLP-3-17 Rev. 01Responsible: Albin BrandsteterAnticipated Completion: 12/31/94

3. Response Approved: Date: I -1 g PM= d - Mnage

RegulAry and Technicafvaluation "MMan-a ga

Exhibit OAP-1 8.1.2 REV. O6/27�94Exhibit CIAP-1 5.1.2 REV. 0127194

- -- 8OFFICE OF CIVILIAN

RADIOACTIVE WASTE MANAGEMENTU.S. DEPARTMENT OF ENERGY

WASHINGTON, D.C.

S.

CAR NO. YM-94-076

PAGE l OF QA


1. Partial Verification of CAR YM-94-076

Reviewed the revised two affected Waste Isolation Evaluations packages associated with Design Package 2C. WE Constructionwater for Package 2C Excavation of the ESF North Ramp DLBABE00000-01717-2200-00008, Rev. 03, and WE Tracers, Fluids,and Materials, and Excavation Methods for Use in the Package 2C Exploratory Studies Facility ConstructionDI:BABE00000-01717-2200-00007, Rev. 03. The WIEs were revised and objective evidence such as hand calculations wereprovided and incorporated to ensure that the calculations used in the WEEs were verified.

Verified that symbolic mathematical expressions similar to those used in the WIEs that were derived by *Macsyma withexpressions given in mathematical handbooks provide similar results.

Verified the output of the algebraic functions calculated by the computer program Macsyma" with the output of the algebraicfunctions calculated by a similar computer program Mathematica.

A A, � I/aleyDateJoh)' Pelletier, QAR

Exhibit OAP-1 8.1.2 REV. 061271S4

Exibit CLP-16.1.2 REV. 06J27194

-- - ~~~~~~~~~~~~~~~~.

'0 OFFICE OF CIVILIAN, I RADIOACTIVE WASTE MANAGEMENT


CAR NO. YM9-7

PAGE L] OF 2QA



A. Remedial:

Objective evidence of calculation verification was not provided in the preparation of the identified Waste IsolationEvaluations (WiEs). Verification checks were performed as good engineering practice, but are not formally documentedunder our current procedures. Such evidence will be provided by incorporating the verification steps described below as anattachement to the affected WIEs:

1. Identify those calculations performed using Macsysma.

2. Comparison of symbolic mathematical expressions simila to those used in the WEEs that were derived by Macsyma withexpressions given in mathematical handbooks.


The above actions are intended to satisf the confidence that the two WIEs are adequate to support Design Package 2Crelease, and do not represent a solution to the longer term issue addressed in the section on Preclude Recurrence.



Waste Isolation Evaluation: Tracers, Fluids, and Materials for Use in the Package 2C Exploratory Studies Facility ConstructionDI: BABE00000-01717-2200-00007 Rev. 00

C. Preclude Recurrence: A revised line procedure (NL-2-0) for Determination of Importance Evaluataions (DIE) is planned.Under the revised line procedure, Waste Isolation and Test Interference Evaluations IE) will be documented as sections ofthe DIE rather than separate documents. The revision of NLP-2-0 consolidating DIE, WIE, and TE procedures willincorporate the requirement for the identification of the verification method(s) used on the results of the use of softwareprograms during the performance of DIEs. Two members of the Performance Assessment and Modeling Departmentreceived training on NLP-19-1 Rev. 00 on 8/10/94.

Develop a classification process to determine if software falls under the QARD Supplement I requirements for Scientific &Engineering (S&E) software. The applicability of S&E software controls on formulas and algorithms that are used as input tonon-S&E software will also be addressed. We shall respond to these issues in revisions of the QAP-19 series procedures.

Ehibit OAP-1t.1.2#h I ,It'1 / r4 -rdji 1491i1n

REV. 06127194

&OFFICE OF CIVILIAN

RADIOACTIVE WASTE MANAGEMENTU.S. DEPARTMENT OF ENERGY

WASHINGTON, D.C.

CAR NO. YM:

PAGE 2 OF 2

QA

CORRECTIVE ACTION REQUEST CONTINUATION PAGE)

2. Response Actions

i1. Revise affected WEEs to incorporate objective evidence of Macsyma calculation verification.

2. Revise QAP-19 series procedures to address classification of software to determine applicability of QARD Supplement Irequirements and applicablity of S&E software controls on formulas and algorithms used as input to non-S&E software.

3. Revise NLP-2-0 to incorporate requirements to identify verification method(s) for results of computer analyses used toevaluate waste isolation or test interference impacts.

Remedial Action

Revision of Package 2C WIEsResponsible: James Houseworth and David SassaniCompleted: 8/19194

Preclude Recurrence

Revision of QAP-19 series proceduresResponsible: Chuck StaffordAnticipated Completion: 12/31/94

Revision of NLP-2-0Responsible: Albin BrandstetterAnticipated Completion: 12/31/94

/ . /4I ZIam4 Date:3. Response Approved: �//7 9V

Regulatory and Tec ale#luation M4nager

I

.... b~ Q 1.1.2 RE. 06127/94............_ ._.__.Exhiblt GA-1B. 1.2 REV. 06/27194

Documents

Department of Energy · A. Remedial: Macsyma is a general purpose program for symbolic and numerical mathematical manipulations that has been widely accepted and used in the scientific