Using Critical Parameter Management to Manage, Analyze & Report Technical Product Performance

1

© 2006, Raytheon Co., All Rights Reserved

Using Critical Parameter Management to Using Critical Parameter Management to Manage, Analyze & Report Technical Manage, Analyze & Report Technical Product PerformanceProduct Performance

Neal Mackertich & Peter KrausNeal Mackertich & Peter Kraus

Raytheon CompanyRaytheon Company

2 © 2006, Raytheon Co., All Rights Reserved

World-Class Systems DFSS World-Class Systems DFSS

Voice of the Customer modeling and analysis an part of the Requirements Voice of the Customer modeling and analysis an part of the Requirements analysis processanalysis process

Up-front Architectural trade space evaluation (vs. validation)Up-front Architectural trade space evaluation (vs. validation)

Statistical modeling & optimization of the performance / cost system Statistical modeling & optimization of the performance / cost system design trade spacedesign trade space

› Utilizing a mathematically linked, collaborative environment for managing and performing Systems design TPM sensitivity analysis

› With CAIV strategically deployed up-front as standard program practice for product lifecycle cost management

Enabled decision-making through predictive defect modelingEnabled decision-making through predictive defect modeling

Stochastically modeled System Integration, Verification & ValidationStochastically modeled System Integration, Verification & Validation


What is Critical Parameter Management?What is Critical Parameter Management?

A disciplined methodology for managing, analyzing, A disciplined methodology for managing, analyzing, and reporting technical product performance.and reporting technical product performance.

A process for mathematically linking system A process for mathematically linking system parameters for sensitivity analysis and optimization parameters for sensitivity analysis and optimization of critical performance threads.of critical performance threads.

A strategic tool for improving product development A strategic tool for improving product development by unifying and integrating systems, software, by unifying and integrating systems, software, design and manufacturing activities.design and manufacturing activities.

CPM = TPMs + Other parameters critical to functionality, cost, schedule or customer


Case for CPM at Raytheon IDSCase for CPM at Raytheon IDS

Raytheon as a Joint Battlespace IntegratorRaytheon as a Joint Battlespace Integrator

Design, management, and performance analysis are becoming increasingly complex and distributed tasks

Mission AssuranceMission Assurance

Increasing our ability to deliver high-performing, affordable systems

Re-use of technical knowledge, analysis tools, and intellectual capitalRe-use of technical knowledge, analysis tools, and intellectual capital

Accelerates the development of future technical leadsAccelerates the development of future technical leads

There are Big $$$ in the Design Performance – Cost Trade SpaceThere are Big $$$ in the Design Performance – Cost Trade Space


Performance AnalysisPerformance Analysis

CPM Statistically Flow-Up Design, Supplier and Manufacturing Capabilities Exposing Performance Risks and Opportunities

X’s

Y’s

Y’sRisk

Opportunity

“The System Can….”


Whitepaper Management and Task DelegationWhitepaper Management and Task Delegation

Engineering Documents and Whitepaper Analyses

Attach Engineering Documents, Models, and Whitepaper Analysis.

Connect people to analyses, requirements, and performance measures.


CPM Primary Program BenefitsCPM Primary Program Benefits

Facilitate AnalysisFacilitate Analysis Statistical modeling & optimization of the performance – cost trade space Real-time System-level sensitivity analysis Connects analyses between system, subsystem and component levels

Improve CollaborationImprove Collaboration Shares technical analysis and knowledge Links ownership to parameters Mathematically connects Program teams and parameters to understand

requirement flow-down Captures and leverages invested intellectual capital for future business

reuse

Streamline ReportingStreamline Reporting

TPM design margins are statistically tracked over product lifecycle Automated, real-time TPM data gathering / report generation Reconciliation of requirement allocation and engineering design capability


DFSS Statistical Requirements AnalysisDFSS Statistical Requirements Analysis

A

B

C

D

E

Y

Y = f (A, B, C, D, E, F,...,M)0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

180

187

194

201

208

215

222

229

236

243

250

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

17

17.6

18.2

18.8

19.4 20

20.6

21.2

21.8

22.4 23

0

0.2

0.4

0.6

0.8

1

1.2

1.4

15

15.8

16.6

17.4

18.2 19

19.8

20.6

21.4

22.2 23

0

0.2

0.4

0.6

0.8

1

1.2

1.4

15

16.5 18

19.5 21

22.5 24

25.5 27

28.5 30

0

0.2

0.4

0.6

0.8

1

1.2

1.4

15

16.5 18

19.5 21

22.5 24

25.5 27

28.5 30

Response

F

G

H

I

J

K

L

M

0

0.05

0.1

0.15

0.2

0.25

15

16.5 18

19.5 21

22.5 24

25.5 27

28.5 30

Design Variables

Allocation/Flow Down


Statistical Performance Analysis Results In…Statistical Performance Analysis Results In…

A prediction of the A prediction of the Response Statistical Response Statistical PropertiesProperties

A prediction of the A prediction of the Probability of Non-Probability of Non-ComplianceCompliance

An assessment of the An assessment of the Contribution of Parameter Contribution of Parameter VariationVariation to Response to Response Performance and CostPerformance and Cost

A

B

C

D

E

Y

f (A, B, C, D, …)

Parameters

Response

F

G

H

I

J

K

L

M

Function

G-Sys. Losses -.45

A-Pavg .35

D-Ant. Eff, .35

F-Integ. Eff. .34

J-Rec. BW -.34

B-Ant. Gain .29

H-Tgt RCS .23

C-Ant. Aperture .21

K-Pulse Width -.19

M-Rec. Out SNR -.15

I-Noise Figure -.12

L-Rep. Freq. -.03

-1 -0.5 0 0.5 1

Measured by Rank Correlation

Certainty is 95.12% from 4.00E+1 to 5.30E+1

.000

.007

.014

.020

.027

0

33.75

67.5

101.2

135

3.75E+1 4.25E+1 4.75E+1 5.25E+1 5.75E+1

Prob(LL<Y<UL)

.000

.007

.014

.020

.027

3.75E+1 4.25E+1 4.75E+1 5.25E+1 5.75E+1

PDF(Y)

Results from Crystal Ball Monte Carlo SW


Radar Subassembly CPM Case StudyRadar Subassembly CPM Case Study

Phased Array RadarPhased Array Radar

High Volume SubassemblyHigh Volume Subassembly

Mechanical Dimensions Critical to Electrical PerformanceMechanical Dimensions Critical to Electrical Performance


Project ObjectivesProject Objectives

Attain Robust Design performance at minimum production Attain Robust Design performance at minimum production cost.cost.

Reduce current unit production cost by 30%.Reduce current unit production cost by 30%.

Aggressively strive for additional cost savings.Aggressively strive for additional cost savings.

Become a documented, successful design phase example Become a documented, successful design phase example for others to follow.for others to follow.


Radar Subassembly Tree StructureRadar Subassembly Tree Structure


Design InterfaceDesign Interface


03.03

;3

00108.006389.0

USLLSLMin C

pk

5004003002001000

0.068

0.067

0.066

0.065

0.064

0.063

0.062

0.061

0.060

sn

GW

Dim B

020406080

100

BinsF

req Dim B

USL(0.064)

LSL(0.062)

Data set with outliers removed Histogram of dimension Dim B relative to Specifications

Use Cpk as a metric of ProcessCapability. Note: Six Sigma qualitylevel = Cpk >1.5.

Dimensional Capability AnalysisDimensional Capability Analysis


X ScorecardX Scorecard

Project Name: Date:

Component CTF (X)Spec Owner UnitsCode LSL Nominal USL Mean Std. Dev. Cp Cpk Sample Size

D1 R admin 0.4250 0.4270 0.4290 0.4270 0.0020 0.3333 0.3333 400D10 R admin 0.4020 0.4030 0.4040 0.4033 0.0020 0.1667 0.1167 400D2 R admin 0.4250 0.4270 0.4290 0.4264 0.0023 0.2899 0.2029 400D3 R admin 0.4020 0.4030 0.4040 0.4009 0.0007 0.4762 -0.5238 400D4 R admin 0.0575 0.0595 0.0615 0.0587 0.0012 0.5556 0.3194 400D5 R admin NA 0.0000 0.0010 0.0012 0.0031 NA -0.0162 400D6 R admin 0.4020 0.4030 0.4040 0.4009 0.0007 0.4762 -0.5238 400D7 R admin 0.4260 0.4275 0.4290 0.4271 0.0015 0.3333 0.2444 400D8 R admin 0.4260 0.4275 0.4290 0.4265 0.0008 0.6250 0.2083 400D9 R admin 0.0620 0.0630 0.0640 0.0639 0.0010 0.3333 0.0367 400

System Level DFSS - Critcial Parameter Management (System Total Component Requirement & X Score Card)

Radiator CPM Example 8/16/2006

Design Specification Validation Test Data


Determined Manufacturing Process Capabilities of existing design dimensions to be Cpk<0.34

Cpk

-0.8-0.6-0.4-0.2

00.20.40.60.8

1

1.21.41.6

Dim1 Dim2 Dim3 Dim4 Dim5 Dim6 Dim7 Dim8 Dim9 Dim10

Dimension

Acceptable

Good

Six Sigma


04.23

109.0531.0

C

pk

LSLUse Cpk metric to establish relativeelectrical performance.

Histogram of electrical output relative to specification

Response Performance Capability AnalysisResponse Performance Capability Analysis

Electrical Performance

0

20

40

60

80

Nu

mb

er

of

Ob

se

rva

tio

ns

LSL


Established Electrical Response Performance capabilities to be Cpk> 1.66

Electrical Performance Capabilities

Max Spec Cpk

-5.84 -3.63 0.35 2.10-5.84 -3.20 0.32 2.69-5.89 -3.03 0.35 2.71-5.95 -2.97 0.41 2.50-6.00 -2.92 0.42 2.43-6.11 -3.01 0.45 2.26-6.22 -3.24 0.52 1.90-6.27 -3.38 0.56 1.73-6.38 -3.36 0.61 1.66-6.44 -3.13 0.62 1.79-6.49 -2.87 0.59 2.04-6.49 -2.48 0.49 2.71-6.49 -2.27 0.45 3.12-6.49 -2.09 0.42 3.51-6.44 -2.01 0.40 3.67-6.38 -2.00 0.39 3.69-6.27 -2.02 0.39 3.56-6.17 -2.05 0.41 3.33-6.00 -2.24 0.42 2.96-5.84 -2.37 0.44 2.58-5.68 -2.69 0.48 2.06


Radar Subassembly Trade Study AnalysisRadar Subassembly Trade Study Analysis

DOE, Regression and Statistical tests of significance identified only DOE, Regression and Statistical tests of significance identified only one design feature to statistically impact performance.one design feature to statistically impact performance.

Utilized gained process capability knowledge and a statistical Utilized gained process capability knowledge and a statistical understanding of the impact of assigned tolerances on performance understanding of the impact of assigned tolerances on performance to trade low-margin mechanical design tolerance for cost to trade low-margin mechanical design tolerance for cost realization opportunities.realization opportunities.

Through a detailed understanding of what drives manufacturing Through a detailed understanding of what drives manufacturing costs, the team was then able to statistically reallocate tolerances to costs, the team was then able to statistically reallocate tolerances to minimize unit production costs.minimize unit production costs.


Attained Six Sigma plus electrical design performance.Attained Six Sigma plus electrical design performance.

Reduced unit production costs by 58%.Reduced unit production costs by 58%.

Achieved cost savings of >$5MAchieved cost savings of >$5M

Achieved follow-on contract cost reductions >$30MAchieved follow-on contract cost reductions >$30M

Radar Subassembly CPM Case StudyRadar Subassembly CPM Case StudyProject ResultsProject Results


Enabling a Collaborative Design EnvironmentEnabling a Collaborative Design Environment

Engineering whitepapers, memos, models, and analyses attach Engineering whitepapers, memos, models, and analyses attach directly to parametersdirectly to parameters

Connect people to analyses, requirements, performance Connect people to analyses, requirements, performance measures, and design trade-spacemeasures, and design trade-space

Serves as real-time “Global Positioning System” for program Serves as real-time “Global Positioning System” for program performance and responsibilityperformance and responsibility


Long Term Collaborative Product Line BenefitsLong Term Collaborative Product Line Benefits

Original Design: Pave Paws

Generation 3: SRP

Platform “A”: Cobra Judy Replacement

Platform “B”: X-Band Radar

Generation 2: UEWR

Product Adaptation: DREX Technology


Traditional TPM Stoplight ReportTraditional TPM Stoplight Report

TPM Number Description Aug '05 Sep '05 Oct '05 Nov '05 Dec '05CDR

Jan '06TPM-XFE-001 Single Pulse Sensitivity G G G G G GTPM-XFE-002 Search Sensitivity G G G G G GTPM-XFE-003 Range Accuracy G G G G G GTPM-XFE-004 Angle Accuracy G G G G G GTPM-XFE-005 RCS Accuracy G G G G G GTPM-XFE-006 Phase Stability Y Y Y Y Y GTPM-XFE-007 Polarization Isolation G G G G G GTPM-XFE-008 Ellipticity Y Y Y Y Y GTPM-XFE-009 Range Sidelobe Level G G G G G GTPM-XFE-010 Range Resolution G G G G G GTPM-XFE-011 2-Way Notch Depth (combined) Y G G G G GTPM-XFE-012 Receive Pattern Sidelobe Level Y G G G G GTPM-XFE-013 Weight G G G G G G

G Meets Requirement with MarginY Meets Requirement with No MarginR Does Not Meet Requirement

Previous TPM Reporting/ Tracking method is vague and ambiguous with respect to the design margin for each metric. TPM report was only

tracked and managed on a monthly basis for the tracking book.


Statistical TPM Management & ReportingStatistical TPM Management & Reporting

s

LSLx

s

xUSLCpk

*3,

*3min

Statistically track design capability and requirement by establishing upper and lower limits

Monitor design capability and requirement convergence over product lifecycle:

- IPDR, PDR, CDR etc.

-Design, Manufacturing, Integration, Test

Probability of Non-Compliance

Cp and Cpk

s

LSLUSLCp

6


New Automated TPM Reporting FormatNew Automated TPM Reporting Format

Meaningful stoplights based on statistical

sensitivity

Compares requirement allocation vs. current

design capability

Includes sensitivity and statistical

information

Captures complete product lifecycle: requirements, design, manufacturing, testing,

validation, etc. (other columns not shown)

TPM design margins are statistically tracked real-time