'

•

CONCORD ASSOCIATES. INC. Systems Peiformance Engineers I CA/TR 93-019-41 _ I

SALEM GENERATING STATION UNITS 1AND2

TECHNICAL EVALUATION REPORT ON THE IPE SUBMITTAL

HUMAN RELIABILITY ANALYSIS

FINAL REPORT

By

P.M. Haas W.E. Gilmore

Prepared for:

U.S. Nuclear Regulatory Commission Office of Nuclear Regulatory Research

Division of Systems Technology

11915 Cheviot Drive Herndon, VA 22070

(703) 318-9262

August, 1995

725 Pellissippi Parkway Knoxville, TN 37932

(615) 675-0930

6201 Picketts Lake Drive Acwonh, GA 30101

(404) 917-0690

ENCLOSURE !.. --- - -- --- ------~--- -- -- - - -- - --·--- ~--96032602-40 960321

PDR ADOCK 05000272 V PDR

..

•

•

CA/TR-93-019-41

SALEM GENERA TING STATION UNITS 1 AND 2

TECHNICAL EVALUATION REPORT ON THE IPE SUBMITTAL

HUMAN RELIABILITY ANALYSIS

FINAL REPORT

By:

P. M. Haas W.E. Gilmore

Prepared for:

U.S. Nuclear Regulatory Commission Office of Nuclear Regulatory Research

Division of Systems Technology

August, 1995

CONCORD ASSOCIATES. INC. Systems Perfomiance Engineers

725 Pellissippi Parkway Knoxville, TN 37932

Contract No. NRC-04-91-069 Task Order No. 41

•

•

----------- -------

TABLE OF CONTENTS

E. EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 E.1 Plant Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 E. 2 Licensee IPE Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 E.3 Human Reliability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

E.3 .1 Pre-Initiator Human Actions . . . . . . . . . . . . . . . . . . . . . . . . 1 E.3.2 Post-Initiator Human Actions . . . . . . . . . . . . . . . . . . . . . . . 2

E.4 Generic Issues and CPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 E.5 Vulnerabilities and Plant Improvements . . . . . . . . . . . . . . . . . . . . . 2 E.6 Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1. INTRODUCTION ............................... -. . . . . . . . . . 5 1.1 HRA Review Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2 Plant Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2. TECHNICAL REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1 Licensee IPE Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.1 Completeness and Methodology . . . . . . . . . . . . . . . . . . . . . 7 2.1.2 Multi-Unit Effects and As-Built, As-Operated Status . . . . . . . . . 7 2 .1. 3 Licensee Participation and Peer Review . . . . . . . . . . . . . . . . . 8

2 .1. 3 .1 Licensee Participation . . . . . . . . . . . . . . . . . . . . . . 8 2.1.3.2 Peer Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2 Pre-Initiator Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2.1 Pre-Initiator Human Actions Considered . . . . . . . . . . . . . . . . 9 2.2.2 Process for Identification and Selection of Pre-Initiator

Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2.3 Screening Process for Pre-Initiator Human Actions - . . . . . . . . . . 11 2.2.4 Quantification of Pre-Initiator Human Actions . . . . . . . . . . . . . 11

2.3 Post-Initiator Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.1 Types of Post-Initiator Human Actions Considered . . . . . . . . . . 14 2.3.2 Process for Identification and Selection of Post-Initiator

Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.3.3 Screening Process for Post-Initiator Response Actions . . . . . . . . 15 2.3.4 Quantification of Post-Initiator Human Actions . . . . . . . . . . . . 15

2.3.4.1 Screening Values . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.3.4.2 Application of ASEP . . . . . . . . . . . . . . . . . . . . . . . 16 2.3.4.3 Estimates of Operator Response Time . . . . . . . . . . . . 17 2.3.4.4 Treatment of Diagnosis . . . . . . . . . . . . . . . . . . . . . 17 2.3.4.5 Performance Shaping Factors Considered . . . . . . . . . . 19 2.3.4.6 Treatment of Dependencies in Post-Initiator Actions . . . . 20 2.3.4.7 Quantification of Recovery-Type Actions . . . . . . . . . . 21

TABLE OF CONTENTS (Cont'd)

2.3.5 Human Actions in the Flooding Analysis ............. -. . . 22 2.3.6 Human Actions in the Level 2 Analysis ....... -. . . . . . . . . . 22 2.3. 7 GSI/USI and CPI Recommendations . . . . . . . . . . . . . . . . . . . 23

2.4 Vulnerabilities, Insights and Enhancements . . . . . . . . . . . . . . . . . . . 23 2.4.1 Vulnerabilities ........................... ; . . . . 23 2.4.2 Insights Related to Human Performance . . . . . . . . . . . . . . . . . 24

2.4.2.1 Results of Sensitivity Analysis . . . . . . . . . . . . . . . . . 24 2.4.2.2 Important Human Actions . . . . . . . . . . . . . . . . . . . . 24

2.4.3 Human-Performance-Related Enhancements . . . . . . . . . . . .. . . 24

3. CONTRACTOR OBSERVATIONS AND CONCLUSIONS ................ 28

4. DATA SUMMARY SHEETS .................................. 31

REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

•

•

E. EXECUTIVE SUMMARY

This Technical Evaluation Report (TER) is a summary of the documentation-only review of the human reliability analysis (HRA) presented as part of the Salem Generating Station Units 1 and 2 (SGS) Individual Plant Examination (IPE) submitted by Public Service Electric and Gas Company (PSE&G) to the U.S. Nuclear Regulatory Commission (NRC). The review was performed to assist NRC staff in their evaluation of the IPE and conclusions regarding whether the submittal meets the intent of Generic Letter 88-20.

E.1 Plant Characterization

The SGS consists of two Westinghouse pressurized water reactor (PWR) units. Both are four-loop plants with large dry containment. Both units have rated power of 3,411 MWt, 1, 115 MWe. SGS-1 began commercial operation in June, 1977; Unit 2, in October, 1981. Hope Creek, a single-unit boiling water reactor, also operated by PSE&G, is located on the same site near Salem, New Jersey. The front end reviewers listed several notable design features that impact core damage frequency. Most significant from a human performance perspective are the ability for bleed-and-feed operation and the manual switchover of the emergency core cooling system (ECCS). Operator action to initiate bleed and feed and to manually switch over ECCS from injection to recirculation mode typically are important operator actions in PWRs for which they are applicable .

E.2 Licensee IPE Process

Utility personnel were involved in the HRA. Walkdowns and documentation reviews were conducted to confirm that the HRA represents the as-built, as-operated plant. The licensee performed an in-house peer review to help assure that the HRA techniques were correctly applied. The HRA approach employed by the licensee addressed both pre-initiator human actions (actions during maintenance, test, etc.) that could cause failure of important equipment on demand during an accident and post-initiator human actions (those taken in response to an accident event). The licensee identified and discussed important human actions. A significant human-performance-related enhancement made was improvement of procedures for identification of loss of coolant accidents outside containment.

E.3 Human Reliability Analysis

E.3.1 Pre-Initiator Human Actions.

The SGS HRA addressed pre-initiator errors in maintenance, test and surveillance actions by incorporating human error into the systems analysis (fault trees) as a specific cause for system unavailability. Both restoration errors and calibration errors were addressed. No numerical screening was employed. Quantification was performed using THERP analysis of four "generic" actions to represent all pre-initiator human actions. The THERP analysis involved

•

•

only very limited assessment of plant-specific performance shaping factors and dependencies, which is a weakness of the approach. However, 115 pre-initiator actions were included in the model as basic events in fault trees, and pre-initiator errors were identified ·by the licensee as among the most important human actions.

E.3.2 Post-Initiator Human Actions.

The post-initiator HRA addressed both response-type and recovery-type actions. The process employed by the licensee to identify and select the post-initiator actions to be quantified included review of procedures and discussion with operations/training staff. No numerical screening was employed to eliminate actions/sequences. Quantification involved selection of coarse screening values believed by the licensee to be conservative, or where necessary to justify lower values, application of the ASEP HRA process for post-initiator actions. This "refined" assessment using ASEP included some assessment of plant-specific performance shaping factors. In many cases, the licensee assumed that because of symptom based procedures and training, diagnosis (cognitive) actions could be ignored. We believe this assumption, as a general statement, to be overly simplistic and non-conservative. Another potential weakness of the licensee's approach is that the analysis took credit for unproceduralized recovery actions. NUREG-1335 guidance is that credit should not be taken for unproceduralized actions without thorough justification. The licensee did not, in our view, provide a thorough justification for crediting the unproceduralized actions. The licensee employed a simplified treatment of dependencies among multiple human actions in a sequence that is generally consistent with other models that have been employed in PRAs.

E.4 Generic Issues and CPI

The licensee addressed Unresolved Safety Issue A-45,' Decay Heat Removal. The front-end reviewers identified several SGS design features significant to the ability to assure decay heat removal, including a) bleed-and-feed, b) an on-site gas turbine as an alternate source of power in loss of ~ffsite power accidents, and c) containment cooler fans that provide an independent method for containment cooling. Human action is required to initiate these features.

The· licensee addressed recommendations of the Containment Performance Improvement (CPI) Program related to the vulnerability of the containment to the potential for hydrogen pocketing. No locations could be identified by the licensee in which hydrogen could accumulate, bum, and lead to equipment or containment failure.

E.5 Vulnerabilities and Plant Improvements

The licensee used importance calculations, sensitivity studies and comparison with other PRAs to screen for vulnerabilities. A vulnerability was defined as a sequence or event that

2

•

passed the systemic sequence reporting criteria in NUREG-1335 and contributed "inordinately" to the CDF in comparison to either a) other core damage sequences or events in the SGS model, orb) similar sequences or events in PRA results for other comparable plants. One vulnerability was identified, related to interfacing system loss of coolant accidents. A procedures deficiency was identified that was common to Westinghouse PWRs and the Westinghouse Owners Group (WOG) Emergency Response Guidelines (ERGs). The ERGs and the SGS Emergency Operating Procedures, which are based on the ERGs, were modified to address this vulnerability.

E.6 Observations

The following observations are pertinent to NRC staffs determination of whether the licensee's submittal meets the intent of Generic Letter 88-20:

(1) The submittal and supporting documentation indicates that utility personnel were involved in the HRA, and that the walkdowns and documentation reviews constituted a viable process for confirming that the HRA portions of the IPE represent the as-built, as-operated plant.

. (2) The licensee performed an in-house peer review that provides some assurance that the HRA techniques have been correctly applied and that documentation is accurate .

(3) The licensee's analysis of pre-initiator human actions was reasonably complete, though simplified and relatively generic. Identification and selection of human actions to be quantified included review of calibration, test and maintenance procedures and discussion with plant personnel. Both calibration and restoration errors were included. Quantification of pre-initiators involved use of THERP to analyze four "generic" pre-initiator actions that represented all (115) pre-initiator actions included in the model. Plant-specific and certainly case-specific analysis was very limited. This weakness, in our view, limits the ability of the licensee to identify factors contributing to human error and therefore plant risk and to identify possible enhancements. However, pre-initiator human errors were identified as among the most important human actions.

(4) The treatment of post-initiator human actions was reasonably complete in scope. The post-initiator HRA included both response-type and recovery-type actions. The process for identification and selection of post-initiator human actions included review of procedures and discussion with plant operations and training staff. Quantification of post-initiator human errors used either coarse screening values or, where warranted for more important actions (to justify lower values), the ASEP process for more detailed calculations. The general ASEP guidance was followed closely by the licensee. Some evaluation of plant-specific performance shaping

• 3

factors was included, consistent with the simplified ASEP process; and, error recovery factors were included according to ASEP guidance. A weakness of the post-initiator HRA, in our view, is that the licensee maintained a simplified and non-conservative view of the impact that symptom based procedures and improved training have had on reducing the need for cognitive action (diagnosis, decision making, etc.) on the part of the operator. Another potential weakness is credit taken for unproceduralized recovery actions. Dependencies among post-initiator actions were treated in a simplified but rational manner not inconsistent with the basic principles of other approaches such as the THERP dependency model.

(5) The licensee identified important human actions through use of importance calculations, sensitivity studies, and qualitative examination of IPE results. The submittal included discussions of important human actions and human-performance­related insights.

(6) The licensee employed a systematic process to screen for vulnerabilities and identify potential enhancements. Vulnerability screening criteria included NUREG-1335 reporting criteria plus a comparison with other PRA results to identify unusual contributors. One vulnerability, associated with loss of coolant outside containment (interfacing system LOCA), was identified, and an enhancement has been implemented.

4

•

•

1. INTRODUCTION

This Technical Evaluation Report (TER) is a summary of the documentation-only review of the human reliability analysis (HRA) presented as part of the Salem Generating Station Units 1 and 2 (SGS) Individual Plant Examination (IPE) submitted by Public Service Electric and Gas Company (PSE&G) to the U.S. Nuclear Regulatory Commission (NRC). The review was performed to assist NRC staff in their evaluation of the IPE and conclusions regarding whether the submittal meets the intent of Generic Letter 88-20.

1.1 HRA Review Process

The HRA review was a "document-only" process which consisted of essentially four steps:

(1) Comprehensive review of the IPE submittal focusing on information pertinent to HRA.

(2) Preparation of a draft TER summarizing preliminary findings and conclusions, noting specific issues for which additional information was required from the licensee, and , formulating requests to the licensee for the necessary additional information.

(3) Review of preliminary findings, conclusions and proposed requests for additional information (RAis) with NRC staff and with "front-end" and "back-end" reviewers

(4) Review of licensee responses to the NRC requests for additional information, and preparation of this final TER modifying the draft to incorporate results of the additional information provided by the licensee and finalize conclusions.

Findings and conclusions are limited to those that could be supported by the document-only review. No visit to the site was conducted. No discussions were held with plant personnel or IPE/HRA analysts, either during the initial review of the submittal, nor after receipt of licensee responses to NRC RAis. No review of detailed "Tier 2" information was performed, except for selected details provided by the licensee in direct response to NRC RAis. In general it was not possible, and it was not the intent of the review, to reproduce results or verify in detail the licensee's HRA quantification process. The review addressed the reasonableness of the overall approach with regard to its ability to permit the licensee to meet the goals of Generic Letter 88-20.

1.2 Plant Characterization

The SGS consists of two Westinghouse pressurized water reactor (PWR) units. Both are four-loop plants with large dry containment. Both units have rated power of 3,411 MWt, 1,115 MWe. SGS-1 began commercial operation in June, 1977; Unit 2, in October, 1981.

5

•

Hope Creek, a single-unit boiling water reactor, also operated by PSE&G, is located on the same site near Salem, New Jersey. The front end reviewers listed several notable design features that impact core damage frequency. Most significant from a human performance perspective are the ability for bleed-and-feed operation and the manual switchover of the emergency core cooling system (ECCS). Operator action to initiate bleed and feed and to manually switch over ECCS from injection to recirculation mode typically are important operator actions in PWRs for which they are applicable .

6

2. TECHNICAL REVIEW

2.1 ·Licensee IPE Process

2.1.1 Completeness and Methodology.

The submittal information on the HRA process was generally complete in scope. Some additional information and clarification was required from the licensee. That information/ clarification was obtained from the licensee in response to an NRC request for additional information (RAI). The HRA approach employed by the licensee addressed both pre-initiator human actions (actions during maintenance, test, etc.) that could cause failure of important equipment on demand during an accident, and post-initiator human actions (those taken in response to an accident event). Pre-initiator human actions were quantified using the Technique for Human Error Rate Prediction (THERP) approach (Ref. 1) Post-initiator human actions were quantified using the Accident Sequence Evaluation Program (ASEP) HRA approach for post-accident actions (Ref. 2) Both response-type actions (anticipated actions in response to an accident event such as those designated in emergency operating procedures), and recovery-type actions (those involving alternative responses or recovery of failed equipment) were addressed. Some limited consideration was given to plant-specific performance shaping factors. The treatment of diagnosis actions was, in our view, very limited. Dependency among multiple actions in the same sequence were addressed.

• 2.1.2 Multi-Unit Effects and As-Built. As-Operated Status.

•

The front end reviewers examined the IPE consideration of events common to the multiple plants located at the site and for systems shared by the plants. Shared systems between Units 1 and 2 are limited to the compressed air, demineralized water, bulk nitrogen supply solid radwaste handling systems. One auxiliary building is used for both units. It houses the control rooms and most of the ex-containment safety systems. A single turbine building is used for both units. The front-end reviewers identified several cases in which manual action for cross-connection and use of shared systems was part of the consideration of credit for use of those systems, including:

• A cross-connection of chilled water system can be made by proper alignment of manual valves. Chilled water is used to supply cooling to the control area coolers and emergency air compressor. Credit for this cross-connection was taken in the flooding analysis.

• A gas turbine located. on site is able to provide an alternate source of onsite power for either Unit 1 or Unit 2 and was credited in the IPE. Manual action, either locally or in the control room is required for actuation of the gas turbine .

7

•

•

The front-end analysts concluded that the IPE included appropriate consideration for multi.­unit effects. We identified no additional significant human performance effects of shared systems that were considered in the licensee's analysis or that should have been considered. The submittal covers the basic approach used to confirm that the IPE development process matched the as-built and as-operated plant. The team sought information primarily from the earlier work covered in the latest update to the Salem Generating Station Probabilistic Risk Assessment (Ref. 3). This information was supplemented with reviews of SGS safety documentation, plant/system drawings, and operating procedures. Plant configurations were also verified by walkdowns of the system hardware. Section 2.4 of the submittal highlights the cut off dates for freezing the plant in its as-built configuration. The front end analysis was frozen in August 1990, while the back end analysis portion was frozen in December 1991. The publication date for the IPE submittal was July 1993. These actions by the licensee helped to assure that the IPE model represents the as-built, as-operated plant.

2.1.3 Licensee Participation and Peer Review.

2.1.3.1 Licensee Participation.

The IPE Submittal, Section 2.1.4, described the basic composition of the study team that participated in the IPE. PSE&G provided overall technical direction and coordination for the effort. These in-house positions were extensively supported by one or more contractors. It is not totally clear how much of the effort was performed by PSE&G and what level was allocated to the contractor organizations. With regard to HRA, the submittal notes that contractor support for the HRA portion of the IPE was provided by PL&G, Inc. Subject matter experts with site-specific knowledge were involved in development ahd review of the HRA.

2.1.3.2 Peer Review.

A relatively extensive peer review is presented in Section 5.0 of the submittal. The review team concentrated their efforts on the following key areas: (1) IPE Process; (2) Initiating Events; (3) Events trees; (4) System Modeling (fault trees); (5) Human factors; (6) Walkdowns; (7) Data; (8) internal flooding; (9) Inter-system LOCA; (10) Computer Codes; (11) Residual Heat removal evaluation; (12) Uncertainty, sensitivity and importance measures; and (13) Results and conclusions. Review team members represented a broad spectrum of expertise and their individual credentials are documented in the submittal. The review method encompassed both a vertical as well as horizontal inquiry into the IPE methodology and its reported findings. The horizontal approach emphasized the completeness of the IPE relative to current industry practice. The vertical approach incorporated "spot checks" on the accuracy of the PRA analytical process. Based on their review, the team generally agreed, that the overall IPE submittal meets the intent of Generic Letter 88-20. However, the team also identified several findings that warranted follow up and response

8

from the originators of the IPE submittal. These findings represented areas where the review team had expressed concerns, or where specific portions of the IPE were judged to be deficient. A summary of the review team findings with regard to the HRA portion of the IPE process are described below:

• Human Errors (Screening Values versus Refined values) - In some cases~ Post­Operator refined HEPs were discovered to be larger than the screening values. In other instances, a degree of conservatism was not appropriately applied to the initial screening values. Without a sufficiently high set of values, significant contributors to the overall risk assessment could be prematurely lost.

• Human Errors (Procedures for Operator Actions) - Key operator actions that were modeled in the IPE analysis were not covered by an applicable plant procedure. This finding was noted for HV AC failure recovery by opening doors, recovery from loss of cooling by cross connecting certain piping. (See Section 2.3.4. 7 of this TER.)

Other than these two comments, the review team found the HRA approach to be "reasonable and acceptable". The IPE originators furnished the review team with a detailed response to each of the above findings. Actions for disposition of each finding were documented. In some cases a decision was made to take issue with a finding, and the rationale for not taking corrective action was also recorded.

• In summary, there is sufficient information in the submittal to conclude that there was a reasonable process for an "in-house" peer review that provides some assurance that the IPE analytic techniques were correctly applied and that the documentation was accurate.

•

2.2 Pre-Initiator Human Actions

Errors in performance of pre-initiator human actions, such as failure to restore or properly align equipment after testing or maintenance or calibration of system logic instrumentation, may cause components, trains, or entire systems to be unavailable on demand during an accident, and thus may significantly impact plant risk. Our review of the HRA portion of the IPE examines the licensee's HRA process to determine what consideration was given to pre-initiator human events, how potential events were identified, the effectiveness of quantitative and/or qualitative screening process(es) employed, and the processes for accounting for plant-specific performance shaping factors, recovery .factors, and dependencies among multiple actions.

2.2.1 Pre-Initiator Human Actions Considered.

The SGS HRA addressed human errors in maintenance, test and surveillance, and calibration by incorporating human error into the systems analysis (fault trees) as a specific cause for

9

•

•

system unavailability. Both restoration (realignment of equipment after maintenance, test or calibration) and- calibration of instrumentation were addressed. The submittal refers to four types of pre-initiator actions, which include restoration errors and miscalibration errors associated with one, two or three instruments.

2.2.2 Process for Identification and Selection of Pre-Initiator Human Actions.

The key concerns of the NRC staff review regarding the process for identification and selection of pre-initiator human events are: (a) whether maintenance, test and calibration procedures for the systems and components modeled were reviewed by the systems analyst(s), and (b) whether discussions were held with appropriate plant personnel (e.g., maintenance, training, operations) on the interpretation and implementation of the plant's test, maintenance and calibration procedures to identify and understand the specific actions and the specific components manipulated when performing the maintenance, test, or calibration tasks.

The submittal does not provide much detail on the process employed to identify and select pre-initiator human actions. It does provide examples of qualitative screening criteria that were used to eliminate certain actions from further consideration (e.g., those human errors that would result in actuation of a control room annunciator/alarm, or those that would be detected by a required post-maintenance/calibration test.) Of particular interest for the SGS analysis was the fact that all pre-initiators are assumed to be represented by one of four "generic" pre-initiator actions corresponding to the four types mentioned above in paragraph 2.2.1, and the fact that pre-initiator human actions were quantified for only a selected set of seven systems. An NRC RAI addressed the general process for identification and selection of pre-initiator actions with emphasis on the qualitative assessment supporting these assumptions which appear to limit the scope and depth of the pre-initiator analysis. The licensee's response to the NRC RAI provides a relatively complete and concise discussion which is summarized/paraphrased in the following paragraphs.

The analysis of pre-initiator human actions was an integral part of, and evolved during, the systems analysis. Each component in the fault trees for front line and support systems was evaluated with respect to the potential for either restoration errors or miscalibration. The evaluation included identification of the test, maintenance, surveillance and calibration needs of each system. These were obtained by review of technical specifications, operating procedures, maintenance manuals, inservice inspection and surveillance documents, plant systems description documents, and discussion with cognizant system engineers. Each pre­initiator human action identified but not eliminated by the qualitative screening noted above was iricluded as a basic event in the fault trees. Some systems had no pre-initiator actions required, and in some systems all pre-initiator actions identified were eliminated by the qualitative screening. An example of an entire system eliminated from further consideration was the Safety Injection System, because a) components automatically realign upon demand, b) no risk-significant calibration actions are taken during plant operation, and c) periodic

10

•

•

surveillance/testing in accordance with technical specifications assures that the system setpoints continue to be correctly calibrated.

The representation of all pre-initiator errors with four generic actions was intended to be a simplified approach to "conservatively envelope" all of the tasks. The licensee examined plant-specific similarities of restoration and calibration tasks, including similarities in procedures, administrative controls and plant practice, and concluded that error probabilities could be adequately (conservatively) represented by the four generic tasks. In our view, a more detailed, case-by-case evaluation of specific restoration and calibration tasks which examines plant-specific and case-specific factors is preferred over the "generic" treatment used by the licensee because the former affords much greater opportunity to gain an understanding of the underlying factors which influence human error and which may be corrected to reduce human error. However, the licensee's assessment did involve a review of the appropriate documentation and discussion with knowledgeable plant personnel; and, the process for identifying and selecting pre-initiator actions appears to have systematically addressed each component/system attempting to identify significant potential pre-initiator human errors.

2.2.3 Screening Process for Pre-Initiator Human Actions.

While the submittal identifies "screening values" for pre-initiator HEPs, the licensee's response to an NRC RAI clarifies that no numerical screening was performed to eliminate less important pre-initiator HEPs from consideration. All pre-initiator errors that were not eliminated by the qualitative screening noted above were included as' a basic event in the IPE models (fault trees). As discussed below in section 2.2.4 the four generic HEPs used to represent pre-initiator human errors were sometimes referred to as "screening" values because they were felt by the licensee to be conservative.

2.2.4 Quantification of Pre-Initiator Human Actions.

The quantification of pre-initiator human actions was performed using the THERP technique and guidance in the Handbook of Human Reliability (Ref. 1). As indicated above, THERP was not used to obtain individual HEP estimates for each operator action. Instead, a THERP analysis was performed for each of the four generic pre-initiator actions defined by the licensee. The "detailed" THERP analysis involved breaking down the generic action into more basic steps and assigning a basic error that corresponds to different generic actions in the Handbook tables. The analysis included construction of THERP trees and calculation of the combined probability of the individual failures following the THERP guidance. The THERP trees were provided in the submittal. Our review of the trees indicated that the licensee's analysis followed the basic structure described in Chapter 20 the Handbook. The trees were formulated in accordance with the prescribed guidance, and numerical estimates were selected from the appropriate tables .

11

•

•

The four generic pre-initiator actions and the estimated HEP values are:

• Miscalibration Errors per calibration = 3.0 X E-3

• Restoration Error per test or maintenance = 5.0 X E-3

• Dependent miscalibration of two instruments = 5.0 X E-4

• Miscalibration of three instruments = 3.0 X E-4.

These basic failure rates furnished the probability inputs for system unavailability using the following formula:

Where:

UA -

p

UA _ (PD'I)(FDT) (INTV)

component unavailability resulting from miscalibration or restoration

miscalibration error/restoration rate

FDT = fault duration time before detection .

INTV = interval between calibration, test, or maintenance

The values calculated from the above formula were incorporated into the fault tree analysis as a specific cause for system unavailability. A total of 115 pre-initiator human errors were incorporated into the IPE model. Each pre-initiator used in the IPE model is comprised of one or more of the basic HEPs from THERP. A table of the unavailability values, including time estimates for FDT and INTV, are presented in Table 3.3.3-6 of the IPE submittal. The results in the table reveal that unavailability due to human error contribution were identified for the following plant systems: (1) Auxiliary Feed Water System (AFS); (2) Chemical and Volume Control System (CVS); (3) Residual Heat Removal System (RHS); (4) Service Water System (SWS); (5) Engineered Safety Features (ESF) System; (6) Auxiliary Building Ventilation System (VAS); and (7) Control Air System (CAS).

In general, we find the relatively generic approach to assessing pre-initiator human actions to be useful for generating a coarse understanding of the quantitative impact of such actions on the CDF. It is somewhat limited in its ability to identify specific factors influencing human

12

•

•

error probability, and therefore to identify potential enhancements which could be made to reduce risk.

An issue addressed in the NRC RAI was the question of the process for determining and verifying the reasonableness of the estimates for the fault duration time, since the ratio of that value to the maintenance/calibration interval can significantly alter the basic HEP. For calibration errors, the fault detection time was taken to be equal to the scheduled calibration interval, which typically is 18 months. This assumes that the miscalibration will not be detected until a scheduled calibration but that all previous errors will be detected upon calibration. The failure probability in this case is equal to the HEP, i.e., the probability of failure on demand, which is a relatively conservative assumption. For restoration errors, the fault detection time was assumed to be equal to the time interval between surveillances, or

. daily or shiftly walkthroughs. This could be a non-conservative assumption, since it implies that every surveillance or walkthrough will be successful in identifying the latent error. The licensee notes that the sensitivity study discussed above in which the HEPs were multiplied by a factor of ten suggests that the effect does not have a major impact on the CDF. For example, assuming that the failure was not detected in ten surveillances would be equivalent to the factor of ten increase in the HEP, which resulted in a 30% increase in the CDF when all restoration errors were increased simultaneously.

A second issue of concern in the NRC RAI was the question of treatment of dependencies in pre-initiator actions. Human factors related dependencies, such as common procedure weaknesses, performance of the task by the same shift, etc., can lead to a much higher overall human error probability for tasks that would otherwise be independent. As noted above, the licensee treated this type of dependency in calibration of two or three instruments directly by estimating a different HEP for those cases compared to calibration of a single (independent) instrument. Another type of dependency is the inter-person dependency between the individual performing the action and a second person checking the task. The submittal noted that such dependencies were treated following the THERP model in Table 20-22 of the Handbook. In the response to the NRC RAI, the licensee provided further information which indicated that the assumptions made in this dependency evaluation (e.g., that the checker used a checkoff list) were verified by review of procedures and by discussions with l&C engineers, maintenance and operations personnel. The licensee's treatment of dependencies in pre-initiator human actions, while not detailed and comprehensive, is generally consistent with the level of detail and completeness typical of other accepted PRAs.

2.3 Post-Initiator Human Actions

Human errors in responding to an accident initiator, e.g., by not recognizing and diagnosing the situation properly or failing to perform required activities as directed by procedures, can have a significant effect on plant risk, and in some cases have been shown to be dominant

13

contributors to core damage frequency (CDF). These errors are referred to as post-initiator human errors. Our review determines the types of post-initiator errors considered by the licensee, and evaluates the processes used to identify and select, screen, and quantify post-initiator errors, including issues such as the means for evaluating timing, dependency among human actions, and other plant-specific performance shaping factors.

2.3. l Types of Post-Initiator Human Actions Considered.

There are two important types of post-initiator actions considered in most PRAs:. response­~ actions, which include those human actions performed in response to the first level directives of the emergency operating procedures/instructions (EOPs, or EOls); and, recovery-type actions, which include .those performed to recover a specific failure or fault (primarily equipment failure/fault) such as recovery of offsite power or recovery of a· · front-line safety system that was unavailable on demand earlier in the event. The SGS HRA addressed both response-type and recovery-type actions per the above descriptions. Relatively simple response-type actions to manually start or align equipment were modeled in the fault trees related to that equipment. Major response-type actions directed by the Emergency Operating Procedures were modeled as top events in .event trees. Recovery actions were applied at the cutset level after initial quantification of the model.

2.3.2 Process for Identification and Selection of Post-Initiator Human Actions.

• The primary thrust of our review related to this question is to assure that the process used by the licensee to identify and select post-initiator actions is systematic and thorough enough to provide reasonable assurance that important actions were not inappropriately precluded from examination. Key issues are whether: (1) the process included review of plant procedures associated with the accident sequences delineated and the systems modeled; and, (2) discussions were held with appropriate plant personnel (e.g., operators, shift supervisors, training, operations) on the interpretation and implementation of plant procedures to identify and understand the specific actions and the specific components manipulated when responding to the accident sequences modeled.

The submittal provides limited direct discussion of the process for identification of human errors to be included in the IPE model. However, there are general statements in a number of discussions in the submittal indicating that procedures were reviewed and that personnel were involved in identification and review of operator actions. The licensee's response to an NRC RAI provides further information on the process. The licensee reviewed previous PRAs to identify potential actions of importance. The licensee further states that

"Development of the system model (event trees and fault trees) included a thorough review of plant specific emergency operating procedures (EOPs), the associated operating procedures (OPs) referenced in the EOPs, and the operator's handbooks

14

and training manuals. This information was supplemented by discussions with plant senior operators to verify, sequence by sequence, that the modeled recovery actions were consistent with procedure. In addition, the analysts conducted a. thorough review of the operating consoles and locations out of control room in which actions would be taken.

This process appears to have provided reasonable assurance that post-initiator actions of major importance were not overlooked.

2.3.3 Screening Process for Post-Initiator Response Actions.

The submittal discussion of quantification of the post-initiator response-type actions, similar to the discussion of pre-initiators, indicated that numerical screening values were employed. However, the licensee clarified in response to an NRC RAJ that no quantitative screening of post-initiator actions was employed. All actions identified from the qualitative review as important were quantified and included in the IPE model. Some of those actions were quantified using "screening" values, or coarse estimates believed by the licensee to be conservative. These screening values were based on guidance in the EPRI SHARP documentation (Ref. 4) modified to account for available time using the Operator Action Trees (OAT) time reliability correlations (Ref. 5). These screening values are discussed further below.

• 2.3.4 Quantification of Post-Initiator Human Actions.

Post-initiator response actions were quantified using either a "refined" calculation following the guidance of the ASEP (Ref. 2) model for post-1.nitiator actions, or using the screening values noted above. The submittal identifies whether the HEP was determined from ASEP or from the screening values.

2.3.4.1 Screening Values. Screening values were selected from a matrix of values, depending on whether the action was deemed to be a "skill-based", "rule-based", or "knowledge-based" action and based on the time required for diagnosis (decision, detection, diagnosis), as follows:

Action Type Time For Diagnosis (Minutes) <5 5 - 30 30 - 120 >120

Skill 0.01 0.001 0.001 0.001 Rule 0.1 0.01 0.01 0.001 Knowledge 1.0 0.3 0.1 0.01

15

•

•

The submittal states that these screening values were "generally" obtained from SHARP (Ref. 4), from OAT (Ref. 5), and from engineering judgment. These numbers, while largely subjectively based, are not inconsistent with coarse estimates used as "conservative" values for final HEPs in other PRAs. Note that they probably would not be appropriate for use as numerical screening values to eliminate human actions or sequences from the model. Typically, a value on the order of 0.5 is used for numerical screening. However, these values were used as final HEPs in the model in lieu of performing a "detailed" HRA. They are not inconsistent with "basic" HEPs (i.e., nominal values unadjusted to account for plant­specific performance shaping factors) suggested for post-initiator actions in THERP and ASEP.

These coarse screening values were used in an initial quantification for all HEPs. Those HEPs that were identified as important/dominant in the initial quantification were refined using the ASEP procedure.

2.3.4.2 Application of ASEP. Refinements to the screening values were undertaken to reduce any undue conservatism introduced in the Screening HRA. The ASEP ·methodology consisted of the following steps:

1. Estimate Maximum Time Available (Tm) for coping with the specified abnormal event.

2. Identify and review the applicable procedures for each operator action.

3. Group the steps that compose each action.

4. Talk through each action with plant training and operations personnel and with a human factors analysis. These "talkthroughs" were performed in the SGS Simulator to determine: .

(a) How the procedure for the specified operator action is entered.

(b) The level of operator training in the use of the EOPs, and the likelihood of an operator to use the EOPs rather than to trust his memory.

(c) Control room staffing during the mitigation of the abnormal events of interest.

( d) Estimates or measurements of the time required to perform the specified operator actions (Ta) .

. (e) Performance shaping factors such as control room labeling .

16

r--

•

•

(t) Recovery factors such as verification of actions performed and control room annunciators.

5. Assess Basic Human Error Probabilities (BHEPs) for the diagnosis and post-diagnosis tasks of each operator action, based on the information obtained in the previous HRA steps.

6. Calculate the estimated final failure probability for each dominant operator action, applying performance shaping factors and recovery factors.

For each action analyzed, the IPE submittal provided a description of the action; an estimate of the maximum available time (Tm); the applicable procedures; descriptions of the action steps, the time needed to perform the step (Ta), and the assessed BHEPs; a description of any diagnosis performed and the associated time (Td) and BHEP, and the total calculated failure probability. Total failure ,probabilities were based on an event tree technique, analogous to the THERP process for constructing operator action trees.

2.3.4.3 Estimates of Operator Response Time. An important factor in determining the HEP was the estimated . time required for operators to perform key actions. The submittal did not discuss the basis for time estimates. In a response to an NRC RAI, the licensee stated that, "The time required by operators to perform a task was, in general, estimated by simulator runs on the SGS training simulator and talk-throughs of the sequence with the senior operators. The estimates attempted, when appropriate, to account for travel, access, action, and second operator checking. In both examples of timing estimates of specific actions requested by NRC and responded to by the licensee, the time estimates were based on operator judgment. Typically, time estimates do involve significant judgment, even when simulator data is available. A concern is that when time estimates are based solely on operator judgment, particularly if those estimates are not made without a structured process or supported by simulator exercises or walk-throughs to provide context, then operator estimates tend to be overly optimistic. In the case of SGS, the operator judgment appears to have been supported by simulator exercises and a reasonably structured interview process in which the context of the operator action was considered.

2.3.4.4 Treatment of Diagnosis. The submittal notes (page 3.3-15) that, "For most of the actions analyzed, there was no diagnosis error included in the total failure probability calculation. This is due to the fact that very little diagnosis is left to the operator. The symptom-oriented (flow-chan) EOPs direct him to the necessary actions based on control room indications. The operator does not need to know what specific abnormal event has occurred in order to get into the proper procedure." While it is true that symptom-based procedures are designed to aid the operating crew in the diagnosis and decision making tasks that are part of accident response, the degree to which these procedures have reduced the likelihood of error in such "cognitive" activities is an open question. Most PRAs assume that

17

•

there still remains a significant error potential, and consequently treat diagnostic error directly. There is little empirical evidence from controlled studies to completely resolve this issue and quantify the likelihood of error. One recently published study (Ref. 6) does· provide some evidence that cognitive functions such as situation assessment and response planning continue to play an important role in accident response, even when symptom-based EOPs are employed.

The ASEP methodology was formulated before there was broad implementation of symptom­based procedures throughout the industry. It should be noted that ASEP guidance (page 8-7) does state that symptom-oriented EOPs may convert formerly knowledge-based behavior such as diagnosis into rule-based behavior; and, that the analyst may judge that the diagnosis aspect of some particular event is negligible because of the combination training and procedures. Thus the licensee's analysis is not inconsistent with the "letter" of the ASEP guidance. However, ASEP guidance cautions that, "In making such a judgment, the analyst must understand that there is a risk of an overly optimistic assessment of human behavior, especially considering the likely stressful nature of abnormal events no one believes will ever occur. . .. Such assessments should be fully documented." ASEP guidance recommends using the lower bound value if and only if:

a) the event is a well-recognized classic (e.g., TMI-2 incident), and the operators have practiced the event in simulator requalification exercises, and

b) the talk-through and interviews indicate that all the operators have a good verbal recognition of the relevant stimulus and know what to do or which written procedures to follow.

ASEP recommends using the nominal value if the only practice of the event is in the simulator requalification exercises and all operators have had this experience.

Review of the HEP calculation summaries (Table 3.3.3-13) in the submittal indicates diagnosis was assumed negligible for a number of actions for which other PRAs have assumed a significant diagnosis component exists, and for which the total HEP would be substantially increased if diagnosis were considered. An example is operator actions to initiate bleed and feed operation. The total time available is 5 to 10 minutes, and the total required action time is 7 minutes. The maximum time available for "diagnosis" is 3 minutes. Another example is failure to initiate boration from the RWST through the BIT, which has a required time of 6 minutes, and a total available time of 10 minutes, with a maximum diagnosis time of 4 minutes.

In our view, the licensee's treatment of the cognitive aspects of operator response employs an overly mechanistic, simplistic, and non-conservative approach to evaluating expected operator performance in the dynamic environment of responding to an accident event. However, the

18

underlying question of the impact that advances in procedures and training (since the TMI-2 event) have had on the operators ability to respond to accident is an open issue. While virtually everyone agrees that these changes have led to improvements, there is little empirical or theoretical basis for quantifying that impact. The licensee's modeling is within the letter of the guidance provided by ASEP, which is an accepted HRA technique that has been used in other accepted PRAs. Further, it appears that the licensee considered each dominant action on a case-by-case basis, rather than simply applying a "blanket" assumption that the diagnosis HEP is always zero. As part of their response to the NRC RAI, the licensee summarized results of sensitivity studies that were performed, in their words, "to examine some of the controversial assumptions made in evaluation of this HEP [both examples - bleed-and-feed and initiation of recirculation]." In both example cases, the licensee presented a more refined assessment of the time available vs. the time required, resulting in an HEP that is roughly an order of magnitude higher than the HEP used in the IPE. The sensitivity studies increased the HEP for both actions by a factor of 10 (apparently independently). The impact of the increase of the bleed-and-feed action was an increase of 4% in the CDP. No new dominant sequences or other insights were identified by the licensee. In the case of initiation of recirculation, the factor of 10 increase in the HEP resulted in a 24% increase in CDP. While these increases in CDP are not large, the underlying issue of the total impact of ignoring the contribution of cognitive error is, in our view, still an open question. We believe greater conservatism in assessing the impact of symptom based procedures is warranted until further quantitative evidence is obtained.

• 2.3.4.5 Performance Shaping Factors Considered. Table 8-5 of the ASEP methodology gives. the analyst an option to introduce performance shaping factors from Chapter 20 of the THERP document if the level of information could be obtained. A stronger and more defendable analysis could be performed if a minimum set of dominant PSFs were at least introduced into the quantification process. For example, in Section 3.3.3.3, the submittal states that, "the control room bezels are generally not well labeled, and that some of the labels do not match descriptions used in the procedures. In addition, many of the pushbuttons are difficult to read when not backlit." It would strengthen the analysis if the negative contributions of these human engineering deficiencies were addressed. The submittal states that the human engineering contributors would not have a significant effect on the final analysis, since the ASEP values are already more conservative relative to other modeling

•

approaches. ·

Another type of performance shaping factors considered in ASEP are error recovery factors. The BHEP may be reduced by credit for discovery and recovery of human error by the individual performing the action or an independent checker. Recovery factors were applied for the majority of procedural steps in accordance with the ASEP methodology. Credit for recovery was taken primarily for a second person who checks the performance of the original performer (e.g., concurrence with operator reading procedure steps). The recovery actions

19

•

e.

were quantified using the ASEP data tables (Table 8-5 of Ref. 2). In these instances, recovery is associated with the aid of an EOP in the performance of control room tasks.

2.3.4.6 Treatment of Dependencies in Post-Initiator Actions.

Two aspects of dependencies were of particular interest for the SGS HRA. First is the treatment of multiple human actions in a cutset. This probably is the most significant dependency issue for most HRAs, because treating multiple dependent actions in a cutset as if they were independent (HEPs simply multiplied) can h_ave a substantial impact on the CDF. The licensee's response states that cutsets with multiple operator actions were assessed to determine the degree of dependence between the multiple actions. The separation or difference of actions in time, in the procedures used~ and in the nature of the action were evaluated to assess whether the actions should be treated as independent or dependent. Multiple independent actions were included in the cutsets with their respective nominal HEP estimates. For dependent actions, the action with the highest HEP was· used, and the other dependent actions were not used. This simplified approach is equivalent to using either "zero" or "complete" dependence in the THERP dependency model. While the THERP model and others used in some PRAs are more complicated, there is little formal basis to justify more precise quantitative models.

A second dependency isstie is addressing the sequence-specific nature of most human actions. In general, every human action is unique and highly dependent on the context in which it takes place. In modeling human actions in event trees, and especially fault trees, it can be difficult to represent this uniqueness. In particular, a concern is that human actions modeled in fault trees will be incorporated in the model as an independent _basic event every time the particular system being modeled is called on, even though the human action may be dependent on other actions and other sequence or cutset-specific factors. Thus the overall contribution of human action to CDF may be significantly underestimated or misinterpreted. The licensee's response to an NRC RAI argues that while this issue is valid theoretically, the practical differences are not risk significant. The licensee's argument is based on several factors: 1) the level of modeling of the human actions is essentially at the same level as the EOPs, which are written for response at a symptom level and are not specific to individual cutsets; 2) the inherent uncertainty in HRA data is such that the case-to-case variation may not be significant; and, 3) the SGS fault trees were developed specifically for the top events of event trees, so that the dependencies with respect to accident sequence groups are, in general, modeled. Item 3 seems to imply that the values used for an HEP in a fault tree are unique to the top event, and that the same action in another fault tree would have a different HEP. This would be an unusual and difficult approach to account for sequence-specific differences. We could not verify the licensee's approach.

Overall, the treatment of dependencies in the SGS HRA is somewhat simplified and limited. However, the analysis does address the issue of dependency among multiple human actions

20

•

on a case-by-case basis and does adjust HEPs to account for identified dependencies. Sequence-specific influences on human error probabilities are at least recognized.

2.3.4.7 Quantification of Recovery-Type Actions. Recovery actions were modeled at the cutset level and incorporated into the IPE model after the initial quantification. The submittal states (page 3.3-14) that recovery actions were quantified, "on a screening basis by use of the matrix [presented above in Section 2.3.4. l]. Others were based on various sources." It is not uncommon practice in HRAs that recovery actions, which are unique to plant equipment, sequence, and even the particular cutset, and which are not routine responses to emergency events, are quantified using essentially operator/analyst judgment. However, it is important to assure that the judgment supported by plant-specific evaluation to assure that the recovery action is feasible and reasonably likely to be taken under the specific accident conditions of concern. In response to an NRC RAI the licensee verified that quantification of recovery actions for SGS was based primarily on subjective estimates supported by plant-specific analysis. The assessment supporting quantification of three specific recovery actions was discussed in some detail. The discussions illustrate that the licensee considered the specific action required, the context of that action, location of the action, control room or other indications/cues indicating the need for action, and other factors influencing human behavior. It is our judgment based on these examples, that in general the licensee conducted a reasonably comprehensive plant-specific assessment in support of quantification of recovery actions. We do note that several non-proceduralized recovery actions were credited, including:

• Open doors and use portable fans following control area or switchgear room HV AC failure (specific HEP not identified).

• Open equipment access hatch in ceiling following failure of diesel generator HV AC (VDG-RCVY-lA, HEP=9E-O, time available > 112 hour).

• Re-connect power to out-of-service SWS pump (SWS-RCVY-BA YTMB, HEP=9E-02).

Section 3.3.7.3 of the submittal notes that credit for the above recovery actions was taken because interviews with operations personnel indicated that such "common sense" actions would be taken. Also Section 5. 8 of the submittal notes that the credited recovery actions appear in certain other EOPs, and consequently plant personnel would be aware of these types of recovery actions. This rationale was challenged earlier by the participants on the peer review. NUREG-1335 suggests that non-proceduralized recovery actions not be credited without thorough justification. We do not consider the justification provided by the licensee to be particularly thorough or convincing. Therefore, credit for these actions should be viewed as a potential weakness of the licensee's approach. We are not able to determine the net quantitative impact of the credit taken for these non-proceduralized actions.

21

•

•

2.3.5 Human Actions in the Flooding Analysis.

There are no specific statements in the submittal regarding the method of quantification of human errors in the flooding analysis. We did identify at least one human action that was quantified. The analysis of flooding in the chiller room determined that rupture of service water system (SWS) piping in that room could flood the room. A human error probability of l.OE-02 was "assumed" for failure to isolate the SWS within the estimated 15 minutes before the components are disabled. The basis for this estimate is not discussed, but appears to be "subjective judgment." A more rigorous assessment is preferred, but it is not uncommon to use subjective estimates such as 0.1 or 0.01 for actions credited in the flooding analysis without the benefit of a detailed analysis. In this case, it is not possible to determine from the information provided in the submittal whether the particular value is or is not "conservative."

2.3.6 Human Actions in the Level 2 Analysis.

The submittal did not _discuss qualitative or quantitative analysis of human actions credited in the Level 2 (back-end) analysis, though several human actions were .credited in the containment event tree (CET). In response to an NRC RAI resulting from the back-end review, the licensee provided a summary description of the basis for quantification of two significant back end actions: ·

• Operator failure in opening PORVs fo high pressure sequences. The back-end reviewer noted that this split fraction can have a significant impact on high pressure melt. ejection, decay heat removal and early containment failure. The licensee noted that this split fraction actually includes both equipment failure (stuck open PORV or SRV) and ·operator failure (intentionally opening a PORV). The operator error probability was assigned nine different values, depending on the specific sequence. The values were based on judgment using the baseline value of 0.5 used .in the Zion NUREG-1150 study (NUREG/CR-4551).

• Operator maintains controlled steam generator cooling after battery depletion in slow station blackout events. The back-end reviewer noted that this action may have a significant impact on the time of vessel breach and on the conditional probability of early containment failure. The estimated HEP of 0.1 was based on discussions with operations personnel about performing this action. No detailed analysis was performed. The licensee notes that a sensitivity study (reported in Section 4.10.3 of the submittal) indicated that this action had a "very minimal impact on overall back­end results."

22

•

2.3. 7 GSI/USI and CPI Recommendations.

Review of the submittal discussions of Generic Safety Issues (GSis) and Unresolved Safety Issues (USis) is primarily the focus of the front-end reviewer. Review of submittal discussions of any licensee actions in response to Containment Performance Improvement (CPI) recommendations is performed primarily by the back-end (Level 2) reviewer. If the licensee's discussion of these issues has particular significance to the HRA or human performance issues, those points are included in this review.

The licensee addressed Unresolved Safety Issue A-45, Diverse Means of Decay Heat Removal (DHR), including use of the power conversion system, bleed-and-feed operation, auxiliary feedwater system, and emergency core cooling system. The front-end reviewer identified three significant design features directly impacting the ability to provide DHR, which were noted previously in the plant characterization (Section 1.2 above). These features included: a) bleed and feed capability, b) an onsite gas turbine, and c) containment cooler fans. Operator action is required for initiation of these systems features. In particular, operator action. to initiate feed-and-bleed operation usually appears as an important action in PWRs having that capability. No unusual human-performance-related issues associated with the licensee's DHR analysis were noted.

The licensee addressed recommendations of the Containment Performance Improvement (CPI) Program related to the vulnerability of the containment to the potential for hydrogen pocketing. No locations could be identified by the licensee in which hydrogen could accumulate, bum, and lead to equipment or containment failure.

2.4 Vulnerabilities, Insights and Enhancements

2.4.1 Vulnerabilities.

The licensee used importance calculations, sensitivity studies and comparison with other PRAs to screen for vulnerabilities. A vulnerability was defined as a sequence or event that passed the systemic sequence reporting criteria in NUREG-1335 and contributed "inordinately" to the CDF in comparison to either a) other core damage sequences or events in the SGS model, orb) similar sequences or events in PRA results for other comparable plants. One vulnerability was identified, related to ISLOCA (summarized as follows in the front-end review). The existing EOPs at the time of the IPE analysis required checking for LOCAs inside containment before considering a LOCA outside containment. At SGS and other Westinghouse PWRs the RHR relief valves direct flow into the pressure relief tank. The blowout of the pressure relief tank rupture disk will indicate a small LOCA. Consequently, operators may transfer to the LOCA procedures and never to the procedure for LOCA outside of containment. This procedure weakness was stated to be common to

23

•

------

Westinghouse PWRs, and a request for evaluation was made to the Westinghouse Owners Group.

2.4.2 Insights Related to Human Performance.

Through the use of importance measures, sensitivity studies, and qualitative review of IPE results, the licensee identified important operator actions and associated insights regarding the contribution of human error to severe accident behavior.

2.4.2.1 Results of Sensitivity Analysis. An example of insight related to human performance was made evident in the sensitivity analysis discussions of the submittal (Section 3.4.2.3). This analysis consisted of eleven sensitivity cases. Each case was quantified by making basic data changes to the case accident sequence cutsets. Two of the sensitivity cases involved analysis on human error events. In Case code INCXHE (i.e., all Service Water Restoration Errors), All XHE events were multiplied by 10. This increase resulted in

. doubling the CDP. While dividing the XHE ~vents by a factor of 10 reduced the CDP by less than 10 % .

2.4.2.2 Important Human Actions. Table 3.4.1-9 in the submittal provides importance measures for the basic events. The licensee used three different importance measures: partial derivative, risk increase, and risk reduction. Tables 2-1 and 2-2 below list the top ten human actions in order of importance using the risk increase and risk reduction measures, respectively. (The partial derivative measure is essentially identical to risk increase, which is more commonly used.) The tables list the ranking (relative importance considering all basic events). Calculated importance values were not reported. Note that pre-initiator errors are among the more important human actions as indicated by both importance measures.

2.4.3 Human-Performance-Related Enhancements.

The vulnerability identified in Section 2.4.1 above was addressed by a procedural enhancement. The Westinghouse Owners Group (WOG) Emergency Response Procedures (ERGs) and the SGS EOPS were modified to include the enhancement. The licensee estimates that the modification reduces CDF by 1 % and large early release frequency by 4%.

Two major changes to the PRA models were identified as a result of the IPE Process. The emergency switchgear ventilation system success criteria have been relaxed (based on a reanalysis of the heat removal calculations) from requiring 2 of 3 supply fans and 2 of 3 exhaust fans at each. elevation to requiring 1 of 3 supply fans, 1 of 3 exhaust fans at the 84 foot elevation, and 1 of 2 of the large exhaust fans at the 64 foot elevation. A second change involves the dependence of the charging pumps on CCS for charging pump seal cooling. Further investigation revealed that the charging pumps could operate without seal cooling.

24

Table 2-1 Ten Most Important Human Actions Ranked by Risk Increase

Designator Action Description HEP Rank

RHS-XHE-FO-RECIR Transfer from injection to recirc mode 5.40E- 30.0 04

ESF-XHE-MC-DF08 Dependent miscalibration of UV sensors 3.00E- 38.0 04

SWS-XHE-RE-12356 Manual valve restoration error 1.37E- 41.5 05

SWS-XHE-RE-12406 Manual valve restoration error 1.37E- 41.5 05

SWS-XHE-FO-XOVER Failure to open header crosstie 1.00E- 65.0 03

SWS-XHE-RE-12134 Manual valve· restoration error 4.llE- 82.0 04

SWS-XHE-RE-12135 Manual valve restoration error 4.llE- 82.0 04

• SWS-XHE-RE-12128 Manual valve restoration error 4.IIE- 82.0 04

AFS-XHE-FO-SGL VL Failure to control steam generator level 1.00E- 110.0 02

AFS-XHE-RE-1MS52 Failure to reset TI valve after test/maint. 5.00E- 111.5 03

25

1--

•

Table 2-2 Ten Most Important Human Actions Ranked by Risk Reduction

Designator Action Description HEP Rank

RHS-XHE-FO-RECIRC Transfer from injection to recirc mode 5.40E- 16.0 04

AFS-XHE-FO-SGLVL Failure to control steam generator level l.OOE- 21.0 02

CCS-XHE-FO-LDISO Failure to isolate non-essential loads l.OOE- 31.0 02

ESF-XHE-MC-DF08 Dependent miscalibration of UV sensors 3.00E- 36.0 04

AFS-XHE-RE-1 MS52 Failure to reset TT valve after test/maint. 5.00E- 37.5 03

AFS-XHE-RE-MSDRN Failure to reset main steam drain valves 5.00E- 37.5 03

CVS-XHE-FO-BORA T Failure to initiate rapid boration 8.00E- 39.0 03

SRV-XHE-FO-FANDB Failure to initiate feed and bleed 4.30E- 60.0 03

V AS-XHE-MC-T7546 Miscalibration of temp. sensor TD-7546 3.00E- 68.5 03

VAS-XHE-MC-T7555 Miscalibration of temp. sensor TD-7555 · 3.00e-03 68.5

Other interesting insights were also noted from the results of the sensitivity analysis for the . Back-End portion of the IPE (Section 7.2.3). The sensitivity analysis identified two findings involving operator/system interaction. The first finding addressed hardware and procedure changes to allow the operators to intentionally open the pressurizer PORVs in all severe accidents. The second finding identified procedure change to allow transfer from LOCA to LOCA Outside Containment EOP. Both conceptual changes represented a significant improvement in accident management.

Footnote bin Table 3.3.3-10 of the submittal, which summarized HEP values, identified several actions for which the quantification took credit for enhancements that were to be made in the future. These included, for example, pre-planning to facilitate action, enhanced

26

•

procedures, pre-staged equipment, and addition of control room annunciation to alert operators. Two apparently significant actions were those associated with providing alternate cooling to the control room area and to the switchgear room in the event of HV AC failures. In response to an NRC RAI, the licensee updated the status of those assumed changes. In some cases, additional calculations have shown that improvements are not required; some procedure changes have been implemented, and some are still in progress .

27

..

•

3. CONTRACTOR OBSERVATIONS AND CONCLUSIONS

The intent of the IPE is summarized in four specific objectives for the licensee identified. in Generic Letter 88-20 and NUREG-1335:

(1) Develop an appreciation of severe accident behavior.

(2) Understand the most likely severe accident sequences that could occur at its plant.

(3) Gain a more quantitative understanding of the overall probability of core damage and radioactive material releases.

(4) If necessary, reduce the overall probability of core damage and radioactive material release by appropriate modifications to procedures and hardware that would prevent or mitigate severe accidents.

With· specific regard to the HRA, these objectives could be restated as follows:

(1) Develop an overall appreciation of human performance in severe accidents; how · human actions can impact positively or negatively the course of severe accidents,

and what factors influence human performance .

(2) Identify and understand the operator actions important to the most likely accident ·sequences and the impact of operator action in those sequences; understand how human actions affect or help determine which sequences are important.

(3) Gain a more quantitative understanding of the quantitative impact of human performance on the overall probability of core damage and radioactive material release. ,

(4) Identify potential vulnerabilities and enhancements, and if necessary/appropriate, implement reasonable human-performance-related enhancements.

The following observations and conclusions are pertinent to NRC staffs determination of whether the licensee's submittal meets the intent of Generic Letter 88-20:

1) The submittal and supporting documentation indicates that utility personnel were involved in the HRA, and that the walkdowns and documentation reviews constituted a viable process for confirming that the HRA portions of the IPE represent the as-built, as-operated plant (at least for the post-initiator error evaluation).

28

•

•

2) The licensee performed an in-house peer review that provides some assurance that the HRA techniques have been correctly applied and that documentation is accurate.

3) The licensee's analysis of pre-initiator human actions was reasonably complete, though simplified and relatively generic. Identification and selection of human actions to be quantified included review of calibration, test and maintenance procedures and discussion with plant personnel. Both calibration and restoration errors were included. No numerical screening was performed; qualitative screening that appears to be rational and consistent with other PRAs eliminated some actions from consideration. All actions surviving the qualitative screening were included in the IPE model as basic events in fault trees. The quantification used THERP to analyze four "generic" pre­initiator actions that represented all (115) pre-initiator actions included in the model. Plant-specific and certainly case-specific analysis was very limited. This weakness, in our view, limits the ability of the licensee to identify factors contributing to human error and therefore plant risk and to identify possible enhancements. However, the analysis appears to have been effective in identifying the relative importance of contributions from pre-initiator human errors.

4) The treatment of post-initiator human actions included both response-type and recovery-type actions. The process for identification and selection of post-initiator human actions included review of procedures and discussion with plant operations and training staff. No numerical screening was employed to eliminate actions or sequences from further consideration. Quantification of human error used either coarse screening values or, where warranted for more important actions (to justify lower values), the ASEP process for more detailed calculations. The ASEP guidance was followed by the licensee. Some evaluation of plant-specific performance shaping factors was included, consistent with the simplified ASEP process; and, error recovery factors were included according to ASEP guidance. The licensee maintained what we believe to be a simplified and non-conservative view· of the impact that symptom based procedures and improved training have had on reducing the need for cognitive action (diagnosis, decision making, etc.) on the part of the operator. Another potential weakness is credit taken for unproceduralized recovery actions. Dependencies among post-initiator actions were treated in a simplified but rational manner not inconsistent with other approaches such as the THERP dependency model.

5) The licensee identified important human actions through use of importance calculations, sensitivity studies, and qualitative examination of IPE results. The submittal included discussions of important human actions and human-performance­related insights.

6) The licensee employed a systematic process to screen for vulnerabilities and identify potential enhancements. Vulnerability screening criteria included NUREG-1335

29

•

reporting criteria plus a comparison with other PRA results to identify unusual contributors. One vulnerability, associated with loss of coolant outside containment (interfacing system LOCA), was identified, and an enhancement has been implemented .

30

• 4. DATA SUMI\1ARY SHEETS

Important Operator Actions/Errors:

The ten most important human actions ranked by risk increase were:

Designator Action Description HEP Rank

RHS-XHE-FO-RECIR Transfer from injection to recirc mode 5.40E- 30.0 04

ESF-XHE-MC-DF08 Dependent miscalibration of UV sensors 3.00E- 38.0 04

SWS-XHE-RE-12356 Manual valve restoration error 1.37E- 41.5 05

SWS-XHE-RE-12406 · Manual valve restoration error 1.37E- 41.5 05

SWS-XHE-FO-XOVER Failure to. open header crosstie 1.00E- 65.0 03

SWS-XHE-RE-12134 Manual valve restoration error 4.llE- 82.0 04 • SWS-XHE-RE-12135 Manual valve restoration error 4.llE- 82.0 04

SWS-XHE-RE-12128 Manual valve restoration error 4.llE- 82.0 04

AFS-XHE-FO-SGL VL Failure to control steam generator level 1.00E- 110.0 02

AFS-XHE-RE-1MS52 Failure to reset TT valve after test/maint. 5.00E- 111.5 03

The ten most important human actions ranked by risk reduction were:

Designator Action Description HEP Rank

RHS-XHE-FO-RECIRC Transfer from injection to recirc mode 5.40E- 16.0 04

AFS-XHE-FO-SGLVL Failure to control steam generator level 1.00E- 21.0 02

CCS-XHE-FO-LDISO Failure to isolate non-essential loads 1.00E- 31.0 02

31

••

•

ESF-XHE-MC-DF08 Dependent miscalibration of UV sensors 3.00E- 36.0 04

AFS-XHE-RE-1MS52 Failure to reset TI valve after test/maint. 5.00E- 37.5 03

AFS-XHE-RE-MSDRN Failure to reset main steam drain valves 5.00E- 37.5 03

CVS-XHE-FO-BORA T Failure to initiate rapid boration 8.00E- 39.0 03

SRV-XHE-FO-FANDB Failure to initiate feed and bleed 4.30E- 60.0 03

V AS-XHE-MC-T7546 Miscalibration of temp. sensor TD-7546 3.00E- 68.5 03

VAS-XHE-MC-T7555 Miscalibration of temp. sensor TD-7555 3.00e-03 68.5

Human-Performance Related Enhancements:

An improvement was made to the EOPs (and to the WOO ERGs) to better address potential LOCAs outside containment (ISLOCA) .

32

REFERENCES

1. Swain, A.D. and H.E. Guttmann, "Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications," NUREG/CR-1278-F, August, 1983.

2. Swain, A.D., "Accident Sequence Evaluation Program Human Reliability Analysis Procedure," Chapter 4, "ASEP Screening HRA for Pre-Accident Tasks," NUREG/CR-4772, February, 1987.

3. Salem Nuclear Generating Station, Probabilistic Risk Assessment (Update), PSE&G and PLG, Inc.; PLG-0792, November 1990.

4. Systematic Human Reliability Procedure CSHARP), Palo Alto, CA: Electric Power Research Institute, June 1984, EPRI NP-3583

5. R.E. Hall, J. Fragola, J. Wreathall, Post Event Human Decision Errors: Qperator Action Tree/Time Reliability Correlation, November 1982, NUREG/CR-3010.

6. E.M. Roth, et al., 11 An Empirical Investigation of Operator Performance in Cognitively Demanding Simulated Emergencies, 11 NUREG/CR-6208, July, 1994.

33