81
T.A. Moebes. 1 Lithium Battery Analysis for Reliability Using Logistic Regression Travis A. Moebes, PhD S&MA/SAIC/JSC [email protected] DRAFT

Lithium Battery Analysis for Reliability Using Logistic … the binomial distribution is used, we might expect that there will be a relationship between logistic regression and chi-square

  • Upload
    builien

  • View
    216

  • Download
    0

Embed Size (px)

Citation preview

T.A. Moebes. 1

Lithium Battery Analysis for Reliability Using Logistic Regression

Travis A. Moebes, PhD S&MA/SAIC/JSC

[email protected]

DRAFT

T.A. Moebes 2

Purpose

Report on BASICS and Methods To Perform Logistic

Regression For Obtaining Probabilities of Failures On Lithium-Ion Battery Cells

Give A Road Map To Follow Good Analysis Practices Through The Steps Of Performing Logistic Regression Using SAS Enterprise Miner (EM) and TIBCO Miner(TM)

Report On A Lithium-Ion Battery Cell Fault Tree And Its Analysis Tied To The Logistic Regression Models For Cell Reliability

T.A. Moebes 3

Topics

The logistic regression model

Testing for the significance of the logistic

regression model – TM and SAS EM

Lithium-Ion Battery Cell Test Data –

Electrovaya, Sony(MoliJ). Motorola (LV)

Logistic Regression and Fault-Tree Analysis of

Cells and Assessment results for Selection &

Reliability – No FTA Applied to Electrovaya

T.A. Moebes 4

The Logistic Regression Model

In logistic regression, you model the probability of a binary event occurring as a linear function of a set of independent variables.

Logistic regression models are a special type of linear model in which the dependent variable is categorical and has exactly two levels.

T.A. Moebes 5

The Logistic Regression Modelcontinued

Linear Regression AnalysisA linear model provides a way of estimating a dependent variable Y , conditional on a linear function of a set of independent variables, X1, X2 ....... Xp. Mathematically, this is written as:

T.A. Moebes 6

The Logistic Regression Modelcontinued

In this equation, the terms are the coefficients βi of the linear model; the intercept of the model is β0 and e is the residual. Estimates of the coefficients, , are computed from the training data from which an estimate of the dependent variable, is computed by substituting the estimated coefficients into Equation for Y above. An estimate of the residual, is then the difference between the observed dependent variable and its estimate.

T.A. Moebes 7

The Logistic Regression Modelcontinued

Logistic Regression Analysis

Logistic regression is a predictive analysis, like linear regression, but logistic regression involves prediction of a dichotomous dependent variable.

The predictors can be continuous or dichotomous, just as in regression analysis, but ordinary least squares regression (OLS) is not appropriate if the outcome is dichotomous.

T.A. Moebes 8

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Whereas the OLS regression uses normal probability theory, logistic regression uses binomial probability theory. This makes things a bit more complicated mathematically, so we will only cover this topic fairly superficially (believe me; I'm mixing it with ease!).

T.A. Moebes 9

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Chi-square and Logistic Regression

Because the binomial distribution is used, we might expect that there will be a relationship between logistic regression and chi-square analysis. It turns out that the 2 X 2 contingency analysis with chi-square is really just a special case of logistic regression, and this is analogous to the relationship between ANOVA and regression

T.A. Moebes 10

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Chi-square and Logistic Regression

With chi-square contingency analysis, the independent variable is dichotomous and the dependent variable is dichotomous. We can also conduct an equivalent logistic regression analysis with a dichotomous independent variable predicting a dichotomous dependent variable.

T.A. Moebes 11

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Chi-square and Logistic Regression - continued

Logistic regression is a more general analysis, however, because the independent variable (i.e., the predictor) is not restricted to a dichotomous variable. Nor is logistic regression limited to a single predictor. Let's take an example. Coronary heart disease (CHD) is an increasing risk as one's age increases. We can think of CHD as a dichotomous variable (although one can also imagine some continuous measures of this). For this example, either a patient has CHD or not. If we were to plot the relationship between age and CHD in a scatter plot, we would get something that looks like this:

T.A. Moebes 12

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Chi-square and Logistic Regression - continued

Figure 1: Plot of CHD by AGE

T.A. Moebes 13

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

We can see from the graph that there is somewhat of a greater

likelihood that CHD will occur at older ages. But this figure is not very suitable for examining that. If we tried to draw a straight (best fitting) line through the points, it would not do a very good job of explaining the data. One solution would be to convert or transform these numbers into probabilities.

Chi-square and Logistic Regression - continued

T.A. Moebes 14

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

The y values can only be 0 or 1, so an average of them will be between 0 and 1 (.2, .9, .6 etc.). This average is the same as the probability of having a value of 1 on the y variable, given a certain value of x (notated as P(y|xi). So, we could then plot the probabilities of y at each value of x and it would look something like this:

Chi-square and Logistic Regression - continued

T.A. Moebes 15

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Chi-square and Logistic Regression - continued

Figure 2 Cumulative Probability Curve For The Logistic Distribution

T.A. Moebes 16

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Chi-square and Logistic Regression - continued

This is a smoother curve, and it is easy to see that the probability of having CHD increases as values of x increases. What we have just done is transform the scores so that the curve now fits a cumulative probability curve for the logistic distribution in Figure2. As you can see this curve is not a straight line; it is more of an s-shaped curve. This s-shape, however, resembles some statistical distributions that can be used to generate a type of regression equation and its statistical tests.

T.A. Moebes 17

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

The Logistic Regression Equation - continued

If we are to get from a straight line (as in regression) to the s-curve (as in logistic) in the above graph, we need some further mathematical transformations.

What we get is an formula with a natural logarithm in it:

This formula shows the relationship between the regression equation (alpha +

beta*x), which is a straight line formula, and the logistic regression equation (the ugly thing on the left). The ugly formula (some twisted folk would say it is beautiful) involves the probability, p, that y equals 1 and the natural logarithm, a mathematical

function abbreviated ln

T.A. Moebes 18

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

The Logistic Regression Equation - continued

In the CHD example, the probability that y equals 1 is the probability of having CHD if you are a certain age. p can be computed with the following formula:

logit transformation

T.A. Moebes 19

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

The Logistic Regression Equation - continued

Logistic regression analysis follows a very similar procedure to OLS regression, only we need a transformation of the regression formula and some binomial theory to conduct our tests.

T.A. Moebes 20

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Model Fit and the Likelihood Function

Just as in regression, we can find a best fitting line of sorts. Instead of minimizing the error terms with least squares, we use a calculus based function called Maximum Likelihood (or ML). ML does the same sort of thing in logistic regression. It finds the function that will maximize our ability to predict the probability of y based on what we know about x. In other words, ML finds the best values for the formulas discussed above to predict CHD with age.

T.A. Moebes 21

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Model Fit and the Likelihood Function - continued

Just as in regression, we can find a best fitting line of sorts. Instead of minimizing the error terms with least squares, we use a calculus based function called Maximum Likelihood (or ML). ML does the same sort of thing in logistic regression. It finds the function that will maximize our ability to predict the probability of y based on what we know about x. In other words, ML finds the best values for the formulas discussed above to predict CHD with age.

T.A. Moebes 22

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Model Fit and the Likelihood Function - continued

ML calculates the fitting function without using the predictor x and then recalculates it using what we know about x. The result is a difference in goodness of fit. The fit should increase with the addition of the predictor variable, x. Thus, a chi-square value is computed by comparing these two models (one utilizing x and one not utilizing x).

T.A. Moebes 23

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Model Fit and the Likelihood Function - continued

The conceptual formula looks like this, where G stands for "goodness of fit":

Mathematically speaking, it is more precisely described as this:

T.A. Moebes 24

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Model Fit and the Likelihood Function - continued

As a result, sometimes you will see G referred to as "-2 log likelihood" as SAS does. G is distributed as a chi-square statistic with 1 degree of freedom, so a

chi-square test is the test of the fit of the model. As it turns out, G is not exactly equal to Pearson chi-square, but it usually lead to the same conclusion.

T.A. Moebes 25

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Odds Ratio and b

As we can see from the logistic equation mentioned earlier, we can obtain "slope" values, b's, from the logistic equation. These, of course, are a result of our transforming equations that allowed us to get

from the logistic equation to the regression equation. The slope can be interpreted as the

change in the average value of y, from one unit of change in x.

T.A. Moebes 26

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Odds Ratio and b - continued

The odds ratio is also obtained from the logistic regression.

It turns out that the odds ratio is equal to the exponential function of the slope, calculated as exp(b). The odds ratio is interpreted as it is with the contingency table analysis. An odds ratio of 3.03 indicates that there is about a three-fold

greater chance of having the disease given one unit increase in x (e.g., 1 year increase in age). If this was the ratio

obtained from the age and CHD example, the odds ratio would indicate a 3.03 times greater chance of having CHD

with every year increase in age.

T.A. Moebes 27

The Logistic Regression Modelcontinued

Logistic Regression Analysis - continued

Odds Ratio and b - continued

It is relatively easy to convert from the slope, b, to the odds ratio, OR with most calculators.

T.A. Moebes 28

Testing for the significance of the logistic

regression model – TM and SAS EM

A t-statistic is a measure of significance for a variable in the model and is the ratio of the coefficient estimate divided by its standard error. In general, a t-statistic greater than 1.96 in magnitude indicates the coefficient that is significantly different from zero and the associated variable should therefore be kept in the model.

TIBCO Miner uses, in part, coefficient estimatesbased on t-statistics and analysis of deviance (R Square) methods for Logistic Regression Model Assessment.

TIBCO Miner

T.A. Moebes 29

Testing for the significance of the logistic

regression model – TM and SAS EM

The p-value for each t-statistic indicates if the corresponding coefficient is significant in the model. In general, if the p-value is less than 0.05 the t-statistic is greater than 1.96. This suggests that the coefficients are significant.

TIBCO Miner- continued

In Figure 3 below, the small p-values for Start implies the term is very significant and the variables Age and Number contribute to the model but less so. Generally,a test for the intercept is uninformative since we rarely expect the regression surface to intersect with the origin.

T.A. Moebes 30

Testing for the significance of the logistic

regression model – TM and SAS EM

TIBCO Miner- continued

TM uses Analysis of Deviance for total model assessment. See Figure 4 below. An Analysis of Deviance table includes the Regression, Error, and Null deviance and the corresponding degrees of freedom (DF). Deviance is a measure of model discrepancy used in logistic regression. As with the Analysis of Variance for a linear model with Gaussian errors the Analysis of Deviance is a partition of a model discrepancy from an oversimplified model, the Null model, into the two components Regression and Error.

T.A. Moebes 31

Testing for the significance of the logistic

regression model – TM and SAS EM

TIBCO Miner- continued

The Null deviance is analogous to the Total sum of squares in an Analysis of Variance. The Regression deviance measures the portion of the Null deviance explained by the model and the Error is what is left over. Further refinement of the Regression deviance by partitioning it by each independent variable is impractical in Data Mining since it would require dropping each variable and refitting the model and taking the difference in the Error deviance. The Term Importance statistic can give a substitute for this refined model

assessment for categorical independent variables.

T.A. Moebes 32

Testing for the significance of the logistic

regression model – TM and SAS EM

TIBCO Miner- continued

From the above hypothetical example with an Analysis of Deviance table we calculate:

R Square (coefficient of determination) = Regression Deviance / Null Deviance = 21.05 / 83.23 =

0.2427.This tells us that 24.27 % of the variation in the

dependent variable is explained by the variation in the independent variables. One should expect at least 50%.

The higher the R Square the better the model. If two models were being compared, we would probably choose

the model with the largest R Square statistic.

T.A. Moebes 33

Testing for the significance of the logistic

regression model – TM and SAS EM

TIBCO Miner- continued

Figure 3 TIBCO Miner Work flow Diagram

T.A. Moebes 34

Testing for the significance of the logistic

regression model – TM and SAS EM

TIBCO Miner- continued

Figure 4 Model Assessment output From TM

T.A. Moebes 35

Testing for the significance of the logistic

regression model – TM and SAS EM

TIBCO Miner- continued

In Figure 6 below, the small p-values for Start implies the term is very significant and the variables Age and Number contribute to the model but less so. Generally,a test for the intercept is uninformative since we rarely expect the regression surface to intersect with the origin.

T.A. Moebes 36

Testing for the significance of the logistic

regression model – TM and SAS EM

SAS Enterprise Miner

SAS Enterprise Miner uses the Likelihood Ratio Test for Global Null Hypothesis to evaluate the entire model. See Figure 6 below. The Null hypothesis is that are no significant variables in the model whose variation explains the variation in the dependent variable. One may reject the Null Hypothesis if Pr is less than 0.0005 (95% confidence) and Beta = 0. Models with higher Likelihood Ratio Chi-Square statistics are the better models.

T.A. Moebes 37

Testing for the significance of the logistic

regression model – TM and SAS EM

SAS Enterprise Miner - Continued

Figure 5 : SAS EM Work Flow Diagram

T.A. Moebes 38

Testing for the significance of the logistic

regression model – TM and SAS EM

SAS Enterprise Miner - Continued

Figure 4: The SAS EM display for the Logistic Regression node

SAS EM also displays t-statistics for term importance with p-values

T.A. Moebes 39

Lithium Battery and Data

Fourteen-hundred rows by 52 columns of Electrovaya Cell Acceptance Data were

obtained from JSC/EP. A portion of the data appears below in Figure 7

Figure 7 Snapshot of the Electrovaya Cell Acceptance Data

T.A. Moebes 40

Lithium Battery and Data - continued

The LIB No. column refers to the battery, the Module No. refers to the Modules containing the cells, and the Serial column refers to the cells. The P/F column indicates if a particular cell (serial number) passed or failed the vendor test.

We assume the possibility of building a Logistic Regression classification model with this data using P/F as the dependent parameter and all other parameters as possible independent parameters.

T.A. Moebes 41

Lithium Battery and Data - continued

Our scope was limited to the Logistic Regression classification for model determination. Other models such as Classification Trees and Classification Neural Networks were not used at this time. Our study was also limited to the above data set. Different data sets and models are planned for the future.

T.A. Moebes 42

Lithium Battery and Data - continued

The Lithium Battery

Figure 8 Cells 11577 (cell 5) and 11657 (cell 4) Damage Due To Debris

T.A. Moebes 43

Lithium Battery and Data - continued

JSC/EP tests showed there are 2 separate internal shorts, both on bottom corners of

cell #11577occurred. The short at GZ affected several cell fold layers more deeply than themore superficial damage caused by short at the opposite corner on cell #11577.

JSC/EP tests showed there are 2 separate internal shorts, both on bottom corners of

cell #11577occurred. The short at GZ affected several cell fold layers more deeply than themore superficial damage caused by short at the opposite corner on cell #11577.

JSC/EP tests showed there are 2 separate internal shorts, both on bottom corners of cell

#11577occurred. The short at GZ affected several cell fold layers more deeply than the moresuperficial damage caused by short at the opposite corner on cell #11577.

T.A. Moebes 44

Analysis and Assessment results

JSC/EP tests showed there are 2 separate internal shorts, both on bottom corners of

cell #11577occurred. The short at GZ affected several cell fold layers more deeply than themore superficial damage caused by short at the opposite corner on cell #11577.

The output of the best model showed good acceptability statistics and an indication of

the failed cell as less acceptable than the other cells.

Data from five cells selected by JSC/EP were processed through three model options

(Option1, Option2, and Option3) to determine the best model and to indicate a known cell

that failed.

All results were similar in both TM and EM processing and model building.

T.A. Moebes 45

Analysis and Assessment results-continued

JSC/EP tests showed there are 2 separate internal shorts, both on bottom corners of

cell #11577occurred. The short at GZ affected several cell fold layers more deeply than themore superficial damage caused by short at the opposite corner on cell #11577.

The output of the best model showed good acceptability statistics and an indication of

the failed cell as less acceptable than the other cells.

Data from five cells selected by JSC/EP were processed through three model options

(Option1, Option2, and Option3) to determine the best model and to indicate a known cell

that failed.

All results were similar in both TM and EM processing and model building.

T.A. Moebes 46

Analysis and Assessment results-continued

JSC/EP tests showed there are 2 separate internal shorts, both on bottom corners of

cell #11577occurred. The short at GZ affected several cell fold layers more deeply than themore superficial damage caused by short at the opposite corner on cell #11577.

Option 2. This option was like Option 1, except six independent variables were dropped because of less significant statistical indications based on p-Values and t-Statistics from Option 1.

Option 1 – All Rows and Columns As Input

A total of 1400 rows of data and all usable columns (53) were used as input to initiate the

process of finding a best model. There were 52 independent variables.

Option 3.

This option was also like Option 1, except deleting the following:· Columns with too many blanks, namely the “soft short test parameters” with the exception of column ACR8C.· Non-relevant columns according to JSC Engineering (EP). Relevant columns with good t-Statistics were kept.· Categorical Columns. These had less than desirable t-Statistics.

T.A. Moebes 47

Analysis and Assessment results-continued

Model Assessment Results

R-Square LR Ratio Pr/Beta

Option 1

0.0750 552.37 0.0001/0.0

Option 2 0.6190 223.49 0.0001/0.0

Option 3 0.6900 787.55 0.0001/0.0

TM SAS EM SAS EM

Figure 8 Model Evaluation Analysis results from both TM and SAS EM.

T.A. Moebes 48

Analysis and Assessment results-continued

Model Analysis Results - TM

Figure 9 Pass/Fail Output From TM and Option 3. Predict.prob is the model prediction that the cell should pass vendor tests. Predict.class is the model indication of Pass or Fail.

T.A. Moebes 49

Analysis and Assessment results-continued

Model Analysis Results – SAS EM

Figure 10 Pass/Fail Output From SAS EM and Option 3. Predict.prob is the model prediction that the cell should pass vendor tests. Predict.class is the model indication of Pass or Fail.

T.A. Moebes 50

Electrovaya Assessment

Our best statistical model showed that cells11577 and 11657 in module FGM-91 and battery 1010 had a probability of 0.79 to pass. v

This was the lowest probability of passing than any other cells in the module. Cell 11657 showed very similar results. All other cells showed a 0.90 probability or higher to pass. Acceptance Statistics were good.

T.A. Moebes 51

Electrovaya Assessment

Further Logistic Regression Models were applied to more Electrovaya Battery Test Cell Data. More low probability indicators were found and confirmed by hardware testingElectrovaya was dropped as the vendor.

Sony (MoliJ) and Motorola (LV) were chosen as the two possible vendors to furnish the Lithium-Ion battery cells.

The next part of the presentation addresses how the LR Model was used in part with Fault Tree Analysis to choose Sony (MoliJ as the vendor.

Goals for MoliJ & LV

Determine a Logistic Regression model to estimate the probability of a cell having a benign internal short based on ABSL/ESD six-sigma standard for passing or failing based on vendor data related to MoliJ and LV cells.

Apply the model to a given collection of MoliJ and LV cells to identify cells (serial numbers) that are most likely to have benign internal shorts, and thus qualitatively most likely not have severe internal shorts.

Estimate the statistical confidence of the models generated in terms of acceptance statistics, such as t-statistics and analysis of deviance for global null hypothesis.

Build and test all models on the TIBCO Miner data mining tools. Select the cells with LM model acceptability less than 95% for identification not to be used in building the battery bricks so that cells chosen have the highest quality against severe internal shorts.

Develop a Fault Tree model and application to estimate the probability of a cell having an internal short based on the Relex calculator and other data base component failure rates. Thus establishing the Reliability of a cell against an internal short.

Logistic Regression Models, Scope, Assumptions, and Data Sources

Cell acceptance data was used in the Logistic

Regression Model.

The independent parameters were the

columns show as mass (weight of an

individual cell), OCV @ BOC (open circuit

voltage at beginning of charge), and others

generated in cell acceptance tests. Thirteen

were chosen out a possible twenty.

Logistic Regression Models, Scope, Assumptions, and Data Sources (continued)

The dependent parameter was the Six-Sigma

acceptability parameter of Pass/Fail. The Pass/Fail

outcome was determined for each cell by all parameters

being in six-sigma range for pass and otherwise for fail

in the train and test data for the Logistic Regression

Model.

Cells that have one parameter out of six-sigma range

are considered by JSC’s ESD to have a benign internal

short that could be an indication of a later severe

internal short.

Logistic Regression Models, Scope, Assumptions, and Data Sources (continued)

Snapshot of the LV Cell Acceptance Data from ABSL

Logistic Regression Models, Scope, Assumptions, and Data Sources (continued)

Snapshot of the MoliJ Cell Acceptance Data from ABSL

Logistic Regression Models, Scope, Assumptions, and Data Sources (continued)

TIBCO Spot Fire Miner Work Flow Diagram

Logistic Regression Models, Scope, Assumptions, and Data Sources (continued)

Data Selection

For the Statistical Risk Assessment (SRA) using Logistic

Regression, 845 rows by 13 columns of ABSL Corporation Cell

Acceptance Data on LV Cells,

And 1048 rows by 13 columns of MoliJ Cells were processed

though Logistic Regression (LR) using TIBCO SpotFire Miner to

find any significant correlation between test output

parameters (independent variables) and the pass/fail outcome

for each of the 845 + 1048 = 1,893 battery cells tested.

Logistic Regression Models, Scope, Assumptions, and Data Sources

(continued)

The pass/fail outcome was determined for each cell by all 13

parameters being in six-sigma range for pass and otherwise for fail in

the train and test data for the Logistic Regression Model. Cells that

have one parameter out of six-sigma range are considered by JSC’s

ESD to have a benign internal short that could be an indication of a

later severe internal short.

One model was made based on and tested on the LV Cells and other

model was made based on and tested on the MoliJ Cells.

Analysis Methodology

Logistic Regression Models, Scope, Assumptions, and Data Sources

(continued)

Snapshot: Determining Six-Sigma Pass/Fail

Analysis Methodology(continued)

Logistic Regression Models, Scope, Assumptions, and Data Sources

(continued)

The train set and test set was the same for each model, respectively.

The output of each model indicated the acceptability of each cell. The

goal was to find helpful predictors for detecting “good” or “bad” cells

in the form of a best Logistic Regression model. A “good” cell was

defined as one with 95% or greater acceptability according to the LR

model and “bad” or not acceptable (for our purposes) otherwise.

Analysis Methodology (CONTINUED)

Logistic Regression Models, Scope, Assumptions, and Data Sources

(continued)

The model determination process was followed by model statistical

acceptance processing. Coefficient estimates, which were based on t-

statistics, and analysis of variance with corresponding probabilities of

test parameters for global null hypothesis, all of which improved the

model determination process and the statistical acceptance

processing.

The output from the best model showed good acceptability statistics

and indicated the failed cell was less acceptable than the other cells

within 95% confidence.

Analysis Methodology(continued)

Logistic Regression Results

Logistic Regression Results (CONTINUED)

Logistic Regression Results (CONTINUED)

Logistic Regression Results (CONTINUED)

Logistic Regression Results (CONTINUED)

During the second set of cycle testing carried out by ABSL, LV cell

#MG13SAAA040 failed during the Performance Based Standards

(PBS) testing. From our 95% confidence cherry picking of the

best cells from ABSL’s initial acceptance testing, we identified 79

cells that are marginal passers. Cell 040 is one of those marginal

passers.

For the MoliJ, we found 87 marginal passers, but ABSL’s PBS

did not identify any true failures.

Our method should be useful in further cherry picking the cells

selected for flight units.

Logistic Regression Results (CONTINUED)

When our list of marginal passers from each lot was sent to

ABSL, ABSL identified how many of each were in the

engineering bricks.

Fault Tree Analysis Models, Scope, Assumptions, and Data Sources

A Fault Tree analysis for an internal short component failures was

developed by JSC\ESD\Battery Group and S&MA\JSC.

Failure rates were obtained from the Relex Calculator and US patent

data bases.

The top event of the fault tree gives the probability of an internal

short failure of a single battery cell.

Fault Trees were run using three different sets of failure rates: 1)

Relex calculator generated with highest quality “D” set; 2) Relex

calculator generated with next lowest quality “mil-std” set; Failure

rates obtained from ABSL paper. All used the same contamination

parameter from the patent database.

Fault Tree Analysis Models, Scope, Assumptions, and Data Sources(continued)

Fault Tree Used To Determine Probability of Internal Short Failure – Quality “D” results shown

Fault Tree Analysis Models, Scope,

Assumptions, and Data Sources (continued)

The top event of the fault tree gives the probability of an internal

short failure of a single battery cell.

Fault Tree Analysis Models, Scope,

Assumptions, and Data Sources (continued)

Component Failure Rates Source

Per 10^6 Hour

Mission Time 5 Years/43800 hrs

Outside Separator 2.46E-07 Relex Calculator

Jelly Roll Top Insulator 2.46E-07 Relex Calculator

Jelly Roll Bottom Insulator 2.46E-07 Relex Calculator

Jelly Roll Outside Insulator 2.46E-07 Relex Calculator

Positive Tab Insulator 2.46E-07 Relex Calculator

Crimp Seal 0.000032

Relex Calculator

Native or FOD Contamination 0.0000014*InfoWorld article

Inside Separator 2.46E-07 Relex Calculator

*fixed probability of failure –

not a failure rate

D Level Highest Quality in Calculations of FRs

Fault Tree Analysis Models, Scope,

Assumptions, and Data Sources (continued)

Component Failure Rates Source

Per Hour

Mission Time 5 Years/43800 hrs

Outside Separator 0.001479 Relex Calculator

Jelly Roll Top Insulator 0.001479 Relex Calculator

Jelly Roll Bottom Insulator 0.001479 Relex Calculator

Jelly Roll Outside Insulator 0.001479 Relex Calculator

Positive Tab Insulator 0.001479 Relex Calculator

Crimp Seal 0.003210 Relex Calculator

Native or FOD Contamination 0.0000001 Patent Database

Inside Separator 0.001479 Relex Calculator

Multiplication Factor = 10E6

Mil-std Next Lowest Quality

in Calculations of FRs

Fault Tree Analysis Models, Scope,

Assumptions, and Data Sources (continued)

Component Failure Rates Source

Per Hour

Mission Time 5 Years/43800 hrs

Outside Separator 0.003456 Relex Calculator

Jelly Roll Top Insulator 0.003456 Relex Calculator

Jelly Roll Bottom Insulator 0.003456 Relex Calculator

Jelly Roll Outside Insulator 0.003456 Relex Calculator

Positive Tab Insulator 0.003456 Relex Calculator

Crimp Seal 0.0001 Relex Calculator

Native or FOD Contamination 0.0000001 Patent Database

Inside Separator 0.003456 Relex Calculator

Multiplication Factor = 10E6

Failure Rates From ABSL

Paper

Fault Tree Analysis Results

Final Results of the Fault Tree

MIL-HDBK-217 FN2 Quality D:

Highest Quality

Probability of Failure for any one cell for Five Years is

1.42 E -006

Odds of Failure = 1 in 704225

Total Failure Probability in 5 years of all Four Bs (at least one cell in each B

fails) =

(320 x 1.42E-006) – ( 1.42E-006)^320 = 4.5E-004 or 1 in 2222

Fault Tree Analysis Results

Final Results of the Fault Tree

MIL-HDBK-217 FN2 Quality C,T:

Second Highest Quality

Probability of Failure for any one cell for Five Years is

.000002

Odds of Failure = 1 in 500,000

Total Failure Probability in 5 years of all Four Bs (at least one cell in each B

fails) =

(320 x 0.000002) – ( 0.000002)^320 = 0.00064 or 1 in 1562

Fault Tree Analysis Results

Final Results of the Fault Tree

MIL-HDBK-217 FN2 Quality S,B:

Third Highest Quality

Probability of Failure for any one cell for Five Years is

.000007

Odds of Failure = 1 in 143,000 (We at least rank here, due to LR Model

Cherry Pick Out Selections )

Total Failure Probability in 5 years of all Four Bs (at least one cell in each B

fails)

= (320 x 0.000007) – ( 0.000007)^320 = 0.00224 or 1 in 446

Fault Tree Analysis Results

Final Results of the Fault Tree

MIL-HDBK-217 FN2 Quality: Mil-std

Next Lowest Quality

Probability of Failure for any one cell for Five Years is

0.000205

Odds of Failure = 1 in 4878

Total Failure Probability in 5 years of all Four Bs (at least one cell in each B

fails) =

(320 x 0.000205) – ( 0.000205)^320 = 0.0656 or 1 in 15

Fault Tree Analysis Results

Final Results of the Fault Tree

FRs From ABSL Paper

Probability of Failure for any one cell for Five Years is

0.000173

Odds of Failure = 1 in 5780

Total Failure Probability in 5 years of all Four Bs (at least one cell in each B

fails) =

(320 x 0.000173 – ( 0.000173)^320 = 0.0656 or 1 in 18

Summary And Conclusions

Conclusion 1: For the SRA and LR modeling, the output from the best model

showed good acceptability statistics and indicated the failed cell was less

acceptable than the other cells within 95% confidence. SRA Analysis identified

21 of the 845 LV cells and 79 of the 1045 MoliJ cells as unacceptable. JSC/ESD

is considering removing these cells from any battery bricks made by ABSL that

presently contain them or do further testing on such bricks. These steps should

increase the acceptability of cells by choosing cells less likely to have benign

internal shorts, and therefore, as well as those less likely to have severe internal

shorts.

Conclusion 2: During the second set of cycle testing carried out by ABSL, LV cell

#MG13SAAA040 failed during the Performance Based Standards (PBS) testing.

From our 95% confidence cherry picking of the best cells from ABSL’s initial

acceptance testing, we identified 79 cells that are marginal passers. Cell 040 is

one of those marginal passers.

Summary And Conclusions (CONTINUED)

Conclusion 3: Both the TIBCO Miner for SRA and the SoHaR RAM

Commander tool for PRA features proved to be helpful in this analysis.

Since both models suggested a high acceptability for the battery cells in

the study. The models are useful for aiding in battery design by

choosing battery cells with a high reliability.

Conclusion 4: For the PRA and Fault Tree Analysis, the probability of

failure for an internal short on a single cell causing failure was

calculated as 1 in 704225 for a mission time of 5 years.