Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
1
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-126MMT L-2 DAY REVIEW
ORBITER PROJECT (MV)sMIA Diode Failure
November 12, 2008
NASASpace
flight.
com
2
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Observation:• During S0024 (hypergolic loading) operations on OV-103
for STS-124, the primary port of multiplexer-demultiplexer (MDM) flight aft 2 (FA2) failed with Non Universal I/O• The failure resulted in a data processing system (DPS)
redundant set split of the general purpose computers (GPCs)• Root cause found to be failed diode on the SMIA (serial
multiplexer interface adapter)
Concern:• Non-universal I/O failure scenarios can range from minor
impacts to engaging the backup flight software (BFS) system
Glen Finneman
NASASpace
flight.
com
3
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Non-Universal I/O Errors: • GPCs communicate with flight critical (FC) MDMs via FC data buses • Each FC MDM has two ‘ports’ connected to FC data buses, one of which is
active at any given time• The active port is selectable via software command• Each port interfaces with the FC bus via the multiplexer interface adapter
(sMIA) • A GPC commands each FC data bus
• While the commanding GPC sends a request on its bus, all other GPCs in the set listen to the returned responses
• Erroneous returns from a commanded unit on the bus are normally seen by all GPCs in the set
• Response errors detected by only a subset of the redundant GPCs are known as “non-universal” I/O errors
• Flight software is designed to declare a GPC that detects non-universal I/O errors as fail-to-sync (FTS)
• Only certain transactions (protected) can cause a Fail-to-Sync (FTS) for Non-universal I/O failures
• Non-universal I/O errors are a documented hazard (accepted risk, remote/catastrophic)
Glen Finneman
NASASpace
flight.
com
4
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 Synopsis:• Intermittent communications between FA2 primary port and
GPC set resulted in GPCs 4 & 2 fail-to-sync (occurred sequentially over several minutes)
• Two of four GPCs with fail to sync called split set by some• GPCs / PASS flight software responded nominally to erratic
communication by removing rogue reporting GPCs• FA2’s secondary port performed nominally when selected• Routine and commanded transactions via the primary and
secondary data buses verified the buses’ integrity• Power cycling FA2 had no effect on the failure condition• Data bus wiring swap, during troubleshooting, isolated the
primary port failure, i.e. primary port remained unresponsive
Conclusion:• MDM FA2’s primary port failed• MDM was R&R’d
STS-124 MDM Failure Investigation
Glen Finneman
NASASpace
flight.
com
5
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
MDM Fault Isolation:• TT&E of removed MDM at NSLD identified a low-
output-voltage failure on the primary port (attenuated signal)
• TT&E data confirmed the serial multiplexer interface adapter (sMIA) is bad• Cause isolated to JANTXV1N3600 diode of lot date code
(LDC) 7715 that had increased forward resistance• Failure Analysis (FA) revealed that the root cause was due
to a cracked dieThe die is the semiconductor material which is the electrically functional part of the diode
Glen Finneman
NASASpace
flight.
com
6
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Diode CR4 Failure Analysis
Figure 1 - viewing through CR4 glass body
Glen Finneman
NASASpace
flight.
com
7
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Diode CR4 Failure Analysis
Figure 2 - view after removal of glass showing diode detail.
Glen Finneman
NASASpace
flight.
com
8
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Diode CR4 Failure Analysis Glen Finneman
NASASpace
flight.
com
9
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Additional Diode Screening Results
Four additional diodes were screened post STS-124:
• Three diodes from the failed sMIA card and one from stores
• note: diode from stores had been used but history unknown
• Electrically screened to paragraph 1.4 of the MIL-S-19500/231 revision that was in effect at time of diode manufacture.
• All diodes passed (data in backup)• Visually inspection all diodes through the glass body
at 100X magnification• All diodes would pass MIL STD-750
• Larger cracks do not appear to be widespread based on this small sample set
Glen Finneman
NASASpace
flight.
com
10
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
1986 – 2471983 – 76021985 - 77151981 - 7535
1994 - 75351984 - ????1989 - 75351985 - 7715
1984 – 82171989 – 7535
1984 – 82172008 – 7715
1986 – 8217
1988 - 8152
No FAOther vendors with FA (no
cracks)
Fairchild other cause
Fairchild cracked
1N3600 PRCA Failure History (Year-Lot Code)
Glen Finneman
NASASpace
flight.
com
11
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Risk Assessment:• sMIA failure scenarios can range from minor impacts
to engaging the backup flight-software system • Flight rules and crew procedures are in place to respond
to a wide range of Avionics failures & their respective criticalities
• For a single MDM/EIU failure, action will be taken to regain capability
• Failure response depends on the phase of flight• MCC flight controllers and crew frequently train non-
universal MDM errors, GPC set splits, and BFS engage scenarios for Ascent and Entry phases of flight
• Operational impacts of these failure scenarios are understood
• Restricting crew interaction with the DPS system• Zero fault tolerant to numerous flight critical systems• Engaging the backup flight system
Glen Finneman
NASASpace
flight.
com
12
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDMFailure Investigation
Risk Assessment:• Hazard report and FMEA/CILs already document effects of a
non-universal I/O error as a program accepted risk (remote/catastrophic) • ORBI 038 “LOSS OF VEHICLE DUE TO FAILURE OF
MULTIPLEXER-DEMULTIPLEXER (MDM) OR ENHANCED MULTIPLEXER-DEMULTIPLEXER (EMDM)"
• Cause B “Erroneous output from (E)MDM-FA1, FA2, FA3, FA4”• This cause documents the effects of erroneous output from an FA
MDM and specifically addresses the effects of non-universal I/O errors• The controls and verifications of non-universal I/O errors are
adequately documented in this cause• FMEA/CILs
• 05-5-B03-1-02 1/1 MDM - FA1, FA2, FA3, FA4 Erroneous Output• 05-5-B03-1A-02 1R/2 EMDM - FA1, FA2, FA3, FA4 Erroneous Output
Glen Finneman
NASASpace
flight.
com
13
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Recurrence Ramifications (cont’d)• Ascent
• Worse case is a GPC Set Split and impacts depend on the severity of the event.
• BFS Engage • No manual flying capability and no single SSME capability.• BFS engage during a black zone is LOCV (near dynamic events such as
SRB separation and MECO or loss of navigational tracking). • Propellant impacts due to manual maneuvers and inertial DAP limitations.
• PASS with two GPC’s.• Potential for loss of Mission success due to system / performance impacts
(i.e. SSME 23k shutdown required or loss of OMS engine functionality).• Zero Fault Tolerance to flight critical GN&C, OMS/RCS, and MPS systems.• Crew procedural actions to safe systems may be limited by reach/visibility.
• Orbit• Worst case is while in Prox Ops a total set split occurs and simplex GPC
is required to regain control.• Actions are dependent on insight into the failure; prox ops case could
result in backout to 250 ft prior to evaluation• If GPC set split is understood (i.e. non-universal error), safe string, then
reconfigure to single GPC and continue.• If GPC set split not understood, potentially delay docking to assess
PASS integrity.
STS-124 MDMFailure Investigation
Glen Finneman
NASASpace
flight.
com
14
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Recurrence Ramifications (cont’d)• Entry
• Worse case is a GPC Set Split and impacts depend on the severity of the event.
• BFS Engage • No auto flying capability and increased crew workload.• Manual speedbrake settings required, no Microwave Landing
System, no pilot beep trim, no approach and landing guidance, • BFS engage during a black zone is LOCV (after extended loss of
navigational tracking). • PASS with two GPC’s.
• Restring required for Vent Door close capability until MM=304.• Zero fault tolerance to flight critical GN&C systems• Crew procedural actions to safe systems may be limited by
reach/vis• Restricted control corridor on the Heading Alignment Circle
(function of atmospheric winds)• Possible no drag chute deployment due to inhibited SSME
repositioning
STS-124 MDM Failure Investigation
Glen Finneman
NASASpace
flight.
com
15
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure InvestigationFigure 1 – OV-105 Partial Flight String
2
1MDMFA1
12
MDMFA3
EIU
1 4321
EIU
2 4321
EIU
3 4321
MEC
1 4321
MEC
2 4321
HUD1 2
1
GPC
3
GPC
4
GPC
5
IDP
1 4321
IDP
2 4321
IDP
3 4321
IDP
4 4321
GPC
1
Flight critical LRUs that contain a sMIA on OV-105
Indicates a sMIA that contain LDC 7715 diodes
GPC
2
Glen Finneman
NASASpace
flight.
com
16
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
OV-105 1N3600 LDCs in a “Protected Transaction” sMIA
FA1 7715 7715 7715 7715 7715 7715 7715 7715FA3 7816 7816 7816 7816 7816 7816 7816 7816
7816 7816 7816 7816 7816 7816 7816 7816
N/A N/A N/A N/A 7816 7816 7816 7816
7715 7715 7715 7715 7816 7816 7816 7815
7715 7715 7715 7715 7816 7816 7816 7818
8112 8112 8112 8112 8112 8112 8112 8112
8812 8812 8812 8812 8812 8812 8812 8812
PORT 1 PORT 3
EIU 3
PORT 2 PORT 4
PORT 2 PORT 4
PRIMARY SECONDARY
PORT 1 PORT 3
PORT 2 PORT 4EIU 1
EIU 2
PORT 1 PORT 3
Note: Green shaded cells have FTS potential
Glen Finneman
NASASpace
flight.
com
17
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
1N3600 SRB Reported Usage Data• STS-126 SRB MDMs
• IOM power supply and core power supply• Criticality 1R• No 7715 or 7535 lot date codes
• MIAs• Criticality 3 only
STS-124 MDMFailure Investigation
Glen Finneman
NASASpace
flight.
com
18
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Flight rationale for STS-126:• 1N3600 diode failure trend is low
• previous failure 19 years ago• total of 17 failures all lots, all vendors for 1N3600
• 4 isolated to cracked Fairchild diodes during FA• UAs or sMIAs without FA could add a few more
occurrences• MDMs are powered on during any vehicle power up
so more likely to see a failure occurrence on the ground than in flight
• OV-105 exposure to the 1N3600 LDC 7715 is limited to two LRUs for STS-126
• MDM FA 1 has this lot on both ports• EIU 2 has this lot on two of four ports
• Only port one has fail-to-sync potential• EIU risk only applies to ascent
STS-124 MDMFailure Investigation
Glen Finneman
NASASpace
flight.
com
19
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Flight rationale for STS-126:• System response to non-universal errors was nominal and
per design• Flight rules document response(s) to comparable failure
during flight• Crew / ground controllers trained to respond to observed
conditions• Documented data processing system functional redundancy
maintained via the backup flight system (BFS) for Ascent/ Entry flight phases
Glen Finneman
NASASpace
flight.
com
20
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Diode Investigation Forward Work• Complete lot date research on all flight hardware
• Priority to hardware that can result in GPC FTS scenario• Other hardware (OI MDMs, other LRUS) as deemed
necessary• Dissect diodes of same lot code
• Understand if observed surface cracks have depth into the die or not
• Determine extent of cracking/crack growth mechanism• Examine parts from prior UA hardware if available• Evaluate options for mitigating risk to future missions
• MDM and EIU R&Rs• SMIA repair (currently no certified vendor)
STS-124 MDM Failure Investigation
Glen Finneman
NASASpace
flight.
com
21
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Backup
NASASpace
flight.
com
22
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Figure 1—GPC to MDM FC Bus Functional Configuration
GPC 1
MDM FF1
MDM FA1
FC1
FC5
P
s
P
S
GPC 3
MDM FF3
MDM FA3
FC3
FC7
P
S
P
S
GPC 2
MDM FF2
MDM FA2
FC2
FC6
P
S
P
S
GPC 4
MDM FF4
MDM FA4
FC4
FC8
P
S
P
S
FC1 + FC5 =
FC String 1
P – Primary PortS – Secondary Port
NASASpace
flight.
com
23
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Figure 3—FA2 (S/N 124’s Anomalous Output from Primary Port)
STS-124 MDM Failure Investigation
NASASpace
flight.
com
24
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Figure 4—FA2 (S/N 124’s Good Output from Secondary Port)
NASASpace
flight.
com
25
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Analysis was completed for all GPC dumps• FA2 I/O errors were erratic with respect to which GPC(s), if
any, encountered an error on any given cycle• In each case, a force Fail to Sync (FTS) occurred as a
result of a single redundant set GPC encountering an FA2 I/O error when the immediately preceding cycle had also encountered I/O errors
• GPC 4 failed first then, within a few seconds, GPC 2 failed• GPC2 was the commander of the FA2 bus• After the GPC 2 FTS, GPCs 1 and 3 were no longer listening to I/O
from FA2 and therefore did not encounter any additional FA2 I/O errors until after a restring was performed to re-assign the FA2 bus to the GPC 1/3 set
• GPC 1 FTS last due to re-introduction of erratic FA2 errors following the restring
• Flight Software performance was per design for the I/O error signature encountered
• The isolation logic attempts to remove a single GPC that is perceived to be preventing successful communication with good end devices (GPC receiver failure for example)
NASASpace
flight.
com
26
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
FSW non-universal I/O error (NUIOE) definition differs from MOD / OPS definition. In FSW logic, a NUIOE is an I/O error logged by one and only one GPC.
FSW design for NUIOEs• Any time an I/O error is detected for an I/O transaction, all GPCs
that are in a redundant set will exchange data about the I/O “seen”by each GPC and:
• IF one and only one GPC detected the I/O error AND an I/O Error was logged by any set of GPCs on the previous instance of the same transaction
• THEN that GPC will be failed from the redundant set (RS Fail-to-Sync).• This logic exists to prevent bypassing a good MDM due to failure of
a GPC receiverFor the STS-124 S0024 incident, dumps were taken for all GPCs
after the set split• Dump data confirmed that this NUIOE logic was the cause of both
GPC 4, GPC 2, and GPC 1 failing from the Redundant Set.• In each case, the non-universal I/O error was associated with an FA2 I/O
transaction• Many I/O errors associated with FA2 transactions were logged during
this period that were seen by more than one GPC.• No indications of off-nominal FSW performance were observed
NASASpace
flight.
com
27
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
NASASpace
flight.
com
28
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Figure 2—Flight Critical 6 Data Bus Physical Configuration
NASASpace
flight.
com
29
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Flight Rule A7-104—Nonuniversal I/O Error Actions
NASASpace
flight.
com
30
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Event History:• During routine hyper loading operations, the DPS was
configured to a four GPC redundant set• At 1206 GMT on 5/13/08, GPC 4 reported an I/O error
associated with MDM FA2• GPC 4 was successfully commanding and listening to
other hardware at the time• At 1212 GMT, GPC 4 detected additional FA2 errors
and was voted out of the redundant set by GPCs 1, 2 and 3
• Seconds later, the remaining GPCs detected FA2 errors albeit inconsistently
• GPC 2 then detected consecutive errors and was voted out of the set by GPCs 1 and 3
• With DPS stable, FA2 was switched to its secondary port and communication was nominal
NASASpace
flight.
com
31
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Event History:• A commanded read of the Master Events Controller’s status
was performed nominally• Verifies the ability of the FA2 primary port data bus to conduct
nominal communications• MDM FA2 was switched back to its primary port where
communication, once again, failed• FA2 was power cycled and its status registers read
• Primary port failed to communicate; secondary port was nominal
• A switch to FA2’s secondary port restored communications• Subsequent troubleshooting swapped data buses at the
MDM• Communication failure stayed with the physical primary port of
the MDM definitively isolating the anomaly to the unit• Analysis of GPC dumps identified the non-universal I/O
errors as the cause of the fail-to-syncs
NASASpace
flight.
com
32
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
HUD2
HUD1
IDP1
IDP2
IDP3
IDP4
MDMFA1
MDMFA2
MDMFA3
MDMFA4
(PLT) (CDR)
FC1FC2FC3FC4
FC1FC2FC3FC4FC5FC6FC7FC8
FC5FC6FC7FC8
MEC1
MEC2
EIU1
EIU2
EIU3
MDMFF1
MDMFF2
MDMFF3
MDMFF4
GPC3
GPC5
GPC4
GPC2
GPC1
FLIGHT CRITICAL DATA BUSSES
NASASpace
flight.
com
33
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Operations....Nominal GPC Configuration
ORBIT
ENTRY
ASCENT
PASSGNC OPS 1
(GNC OPS 6)
RUN
PASSGNC OPS 1
(GNC OPS 6)
RUN
PASSGNC OPS 1
(GNC OPS 6)
RUN
PASSGNC OPS 1
(GNC OPS 6)
RUN
BFSGNC OPS 1,
(6 & 3)
RUN
BFSGNC OPS 1,
(6 & 3)
SLEEP
PASSSM OPS 2
RUN
FREEZE DRYPASS
GNC OPS 2
SLEEP
PASSGNC OPS 2
RUN
PASSGNC OPS 2
RUN
PASSGNC OPS 3
RUN
PASSGNC OPS 3
RUN
PASSGNC OPS 3
RUN
PASSGNC OPS 3
RUN
BFSGNC OPS 3,
(1 & 6)
RUN
NASASpace
flight.
com
34
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
GPC Set SplitWhen GPCs stop talking to each other
Each GPC maintains command of its own string(s)GPCs in RS bypass (no longer listen to) strings commanded by GPCs outside of RS
GPCGPCGPCGPC GPCGPCGPC GPC
GPC GPC GPC GPC
1. Redundant Set of 4 2. GPC X Fail-To-Sync
GPCGPC GPC GPC
4. “2 on 1 on 1” Set Split
GPCGPC GPCGPC
3. “2 on 2” Set Split
5. Total Set Split 6. Cascade
Progression fromexample
1 to 2/3 to 4,
and in some cases to 5
NASASpace
flight.
com
35
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEWShuttle Data Processing
System GPC-MDM Communication Perspective
I/O Transaction:• Commander polls MDM; MDM Responds to Commander• All GPCs in RS listen for MDM response• GPCs in Redundant Set synchronize, reporting joy/no joy on transaction
to each other• BFS also “tracks” PASS by listening to MDM response and status from
PASS. BFS can only track PASS set with a minimum of 2 good strings.I/O Error Handling In a Redundant Set:
• Universally Detected I/O Error – An I/O Error where all GPCs in the Redundant Set (RS) report an error on an MDM.
• Bypass – All GPCs stop listening to responses from MDM after 2nd failed attempt of an I/O transaction if errors are universally detected.
• Non-Universal I/O Error – An I/O Error where some GPC(s) in the RS report an error on a transaction while other GPC(s) do not report an error.
• Force-Fail-To-Sync Due To Non-Univ I/O Errors – One GPC in the RS detects I/O error while all other GPCs do not detect error on the 2nd try of an I/O transaction. The computer that detected the error fails to sync as a result of this non-universal I/O error.
NASASpace
flight.
com
36
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
MDM S/N 124 Discrepant SMIA Card (S/N 1132)
NASASpace
flight.
com
37
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
CR1 Screening Results
Diode Identification: CR1 – From “Stores” (origin unknown) Test Date: 10/23/08 Primary Electrical Characteristics
MIL-S-19500/231, Para. 1.4 Marginal Forward Voltage (VF) Evaluation
Characteristic Measured
Measured (VDC)
Pass/Fail <5 mV Below Max (Yes/No)
IF Applied (mA)
VF Meas. (VDC)
VF1 IF = 1.0 mA
Min: 0.540 VDC Max: 0.620 VDC
0.5798 Pass No n/a n/a
VF2 IF = 10.0 mA
Min: 0.660 VDC Max: 0.740 VDC
0.6896 Pass No n/a n/a
VF3 IF = 50.0 mA (Pulsed)
Min: 0.760 VDC Max: 0.860 VDC
0.778 Pass
VF4 IF = 100.0 mA (Pulsed)
Min: 0.820 VDC Max: 0.920 VDC
0.825 Pass
VF5 IF = 200.0 mA (Pulsed)
Min: 0.870 VDC Max: 1.000 VDC
0.881 Pass
IR1 VR = 50.0 VDC
Min: ---- Max: 0.1 uA
0.019 uA Pass
Visual and Mechanical Examination
Ref: MIL-STD-750 Cracks:
No visible cracks.
Notes: The “C” spring contact is poorly aligned with the PN Junction contact button – Misses the button by more than 50%. Reference MIL-STD-750, Sect. 2074.5, Appendix A – This is NOT a cause for reject.
NASASpace
flight.
com
38
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
CR1 Screening Results
NASASpace
flight.
com
39
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
CR2 Screening Results
Diode Identification: CR2 – From sMIA, s/n 1132 Test Date: 10/23/08 Primary Electrical Characteristics
MIL-S-19500/231, Para. 1.4 Marginal Forward Voltage (VF) Evaluation
Characteristic Measured
Measured (VDC)
Pass/Fail <5 mV Below Max (Yes/No)
IF Applied (mA)
VF Meas. (VDC)
VF1 IF = 1.0 mA
Min: 0.540 VDC Max: 0.620 VDC
0.5796 Pass No n/a n/a
VF2 IF = 10.0 mA
Min: 0.660 VDC Max: 0.740 VDC
0.6902 Pass No n/a n/a
VF3 IF = 50.0 mA (Pulsed)
Min: 0.760 VDC Max: 0.860 VDC
0.780 Pass
VF4 IF = 100.0 mA (Pulsed)
Min: 0.820 VDC Max: 0.920 VDC
0.825 Pass
VF5 IF = 200.0 mA (Pulsed)
Min: 0.870 VDC Max: 1.000 VDC
0.890 Pass
IR1 VR = 50.0 VDC
Min: ---- Max: 0.1 uA
0.021 uA Pass
Visual and Mechanical Examination
Ref: MIL-STD-750 Cracks:
Appears to have several small cracks along three edges of the die. They do not appear to be contiguous and do not appear to transect the die.
Notes:
NASASpace
flight.
com
40
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
CR2 Screening Results
NASASpace
flight.
com
41
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
CR3 Screening Results
Diode Identification: CR3 – From sMIA, s/n 1132 Test Date: 10/23/08 Primary Electrical Characteristics
MIL-S-19500/231, Para. 1.4 Marginal Forward Voltage (VF) Evaluation
Characteristic Measured
Measured (VDC)
Pass/Fail <5 mV Below Max (Yes/No)
IF Applied (mA)
VF Meas. (VDC)
VF1 IF = 1.0 mA
Min: 0.540 VDC Max: 0.620 VDC
0.5845 Pass No n/a n/a
VF2 IF = 10.0 mA
Min: 0.660 VDC Max: 0.740 VDC
0.6987 Pass No n/a n/a
VF3 IF = 50.0 mA (Pulsed)
Min: 0.760 VDC Max: 0.860 VDC
0.793 Pass
VF4 IF = 100.0 mA (Pulsed)
Min: 0.820 VDC Max: 0.920 VDC
0.830 Pass
VF5 IF = 200.0 mA (Pulsed)
Min: 0.870 VDC Max: 1.000 VDC
0.890 Pass
IR1 VR = 50.0 VDC
Min: ---- Max: 0.1 uA
0.018 uA Pass
Visual and Mechanical Examination
Ref: MIL-STD-750 Cracks:
There are no visible cracks in this die. Notes:
NASASpace
flight.
com
42
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
CR3 Screening Results
NASASpace
flight.
com
43
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
CR5 Screening Results
Diode Identification: CR5 – From sMIA, s/n 1132 Test Date: 10/23/08 Primary Electrical Characteristics
MIL-S-19500/231, Para. 1.4 Marginal Forward Voltage (VF) Evaluation
Characteristic Measured
Measured (VDC)
Pass/Fail <5 mV Below Max (Yes/No)
IF Applied (mA)
VF Meas. (VDC)
VF1 IF = 1.0 mA
Min: 0.540 VDC Max: 0.620 VDC
0.5787 Pass No n/a n/a
VF2 IF = 10.0 mA
Min: 0.660 VDC Max: 0.740 VDC
0.6905 Pass No n/a n/a
VF3 IF = 50.0 mA (Pulsed)
Min: 0.760 VDC Max: 0.860 VDC
0.782 Pass
VF4 IF = 100.0 mA (Pulsed)
Min: 0.820 VDC Max: 0.920 VDC
0.822 Pass
VF5 IF = 200.0 mA (Pulsed)
Min: 0.870 VDC Max: 1.000 VDC
0.894 Pass
IR1 VR = 50.0 VDC
Min: ---- Max: 0.1 uA
0.020 uA Pass
Visual and Mechanical Examination
Ref: MIL-STD-750 Cracks:
There appears to be a small crack along one edge of this die.
Notes:
NASASpace
flight.
com
44
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
CR5 Screening Results
NASASpace
flight.
com
45
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Risk Assessment:• Crew and MCC response to GPC set split
• Crew response• On ascent and entry, crews will engage BFS for a total set split and
may engage for 2-1-1 or 2-2• If vehicle control is unstable, the crew will engage BFS
• On orbit crew would only engage BFS for total loss of PASS• MCC response
• Ascent: decision to engage BFS when it is not tracking PASS must be made quickly
• ~10 seconds is the limit for a successful BFS engage• SSME securing actions not taken by MECO could result in
catastrophic engine shutdown in 2-1-1 case• Entry: additional time to engage BFS when it is not tracking PASS
• ~45 seconds is the limit for a successful BFS engage• Orbit: critical time is during rendezvous prox-ops
• If GPC set split is understood (i.e. non-universal error), safe string, then reconfigure to single GPC and continue
• If GPC set split not understood, potentially delay docking to assess PASS integrity
NASASpace
flight.
com
46
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
Risk Assessment: • Hierarchy of Potential Impacts from a GPC set split
Loss of system insightManual system reconfiguration
Loss of system insight
Manual system reconfiguration
Systems zero fault-tolerance
Systems zero fault-tolerance
Loss of system insightLoss of mission due to performance impacts.
BFS EngageManual reconfigurationBFS Engage
Loss of crew/vehicle (loss of control)
Loss of Crew/Vehicle (Collision or Damage to ISS from plume)
Loss of crew/vehicle (loss of control)
EntryOrbitAscent
Dec
reas
ing
Sev
erity
NASASpace
flight.
com
47
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation1N3600 Failure History
1Y19897535FAIRCHILD STRAT
A12186
1N1986247TELEDYNE MICRO
AD1513
1Y19857715FAIRCHILD STRAT
AD2380
1Y19888152AHONEYWELL-SATEL
AD4772
1N19897535FAIRCHILD STRAT
AD5376
1Y20087715FAIRCHILD STRAT
NA2095
1Y19857715FAIRCHILD STRAT
AD2382
1N19947535NA0372
1Y19868217UNITRODEAD1439
1Y19848217BUNITRODEAC9027
1Y19848217UNITRODEAD2221
1Y1984.????HONEYWELL-SATEL
AC7973
1Y19837602ZZHONEYWELL-SATEL
AC6555
1Y19817535FAIRCHILD STRAT
AC0935
OtherCracked Die
Failure Analysis
Detected Year
Lot Date CodeManufacturerCAR#
NASASpace
flight.
com
48
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
STS-124 MDM Failure Investigation
OV-103 (LON) sMIA 1N3600 LDCs
FA4 7715 7715 7715 7715 7501 7501 7501 7501
all others N/A N/A N/A N/A N/A N/A N/A N/A
N/A N/A N/A N/A N/A N/A N/A N/A
N/A N/A N/A N/A N/A N/A N/A N/A
8112 8112 8112 8112 8112 8112 8112 8112
8112 8112 8112 8112 8112 8112 8112 8112
N/A N/A N/A N/A N/A N/A N/A N/A
N/A N/A N/A N/A N/A N/A N/A N/A
PORT 1 PORT 3
EIU 3
PORT 2 PORT 4
PORT 2 PORT 4
PRIMARY SECONDARY
PORT 1 PORT 3
PORT 2 PORT 4EIU 1
EIU 2
PORT 1 PORT 3
NASASpace
flight.
com
49
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Failure History Search
Numerous Fairchild 1N3600 diodes were found in PRACA search
• CAR NA2095 – MDM sMIA – LDC 7715 – cracked die - current failure
• CAR AD2380 – MDM CPS – LDC 7715 – cracked die (1985) • CAR AC0935 – MDM CPS – LDC 7535 – cracked die (1981)• CAR AD5376 – MDM CPS – LDC 7535 – no failure analysis
performed-(1989)• CAR NA0372 – MDM CPS – LDC 7535 – no failure analysis
performed no manufacturer identified, part number listed as “FJHTXV3600” (1994)
• CAR A12186 – SRB MDM MIA – LDC 7535 cracked die “amplitude of response data word was ramping between 7.5 and 11.9 Vdc and below the specified limit of 12.0Vdc”; 1606 other diodes of this manufacturer were visually inspected and 2 diodes(LDC 7535 and 8038) had cracks through the semiconductor construction” (1989)
• Fits description of the failure mode seen on NA2095
NASASpace
flight.
com
50
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Failure History (cont)
• UA CAR AD6071 Orbiter MDM Primary IOM PS- Hybrid latched up in current fold-back state, Hybrid and 6 Diodes were removed as probable cause of the failure, all diodes tested to spec, hybrid failed at 100 C. No FA on Hybrid or diodes. Diodes listed as PN: 4004605-601 LDC 7535. (1989)
• UA CAR A12372 SRB MDM MIA – primary output voltage measured was at 11.6 Vdc, no failure analysis of the card, probable cause was transistor? (1989)
• UA CAR A11858 SRB MDM MIA- primary output voltage was 11.279VDC, MIA replaced MDM passed ATP, Failure on MIA not reproduced, MIA limited to ground use only. (1988)
• UA CAR A12160 SRB MDM MIA- data response distorted and decaying to an amplitude of ~7VDC during each response word. Troubleshooting of MIA found to be below spec but no longer repeating 7V decay. Out of spec resistors and transistor epoxy die attach could be the cause. MIA not repaired and will not be flown. (1989)
• UA CAR A00584 SRB MDM MIA- Failed to communicate, problem was isolated to the MIA, MIA returned to Singer/Plessey for failure analysis, two likely causes the MIA or a crimp connection on the data buss connector with the most likely cause being the MIA. (1979)
• UA CAR 12268 SRB MDM MIA- Failed when tested hot, only probable cause was Data Bus Failure of MIA, MIA returned to Singer/Plessey for failure analysis, could not duplicate problem, MIA limited to ground use only. (1989)
NASASpace
flight.
com
51
Organization/Date:
Presenter:
Orbiter / 11-12-08
Pre-decisional. Internal Use Only
126lpSTDiode 11/18/2008 12:00 pm
STS-126 MMT L-2 DAY REVIEW
Worst-case Scenario for sMIA Diode Failure & Blackzone Applicability from DPS/MOD Jennifer McDonald
For the non-universal I/O error cases, where the end result is no set of at least two PASS GPC remaining (“cascading set split” is MOD term for this series of force fail-to-syncs), the BFS will be Standalone.
• After the initial non-universal I/O errors and the resulting single GPC force-fail-to-sync or set split (2-on-2 or 2-on-1-on-1) , one or two more non-universal I/O event(s) are seen by the remaining GPCs in the redundant set within a few seconds
• Result would be the total loss of the PASS Set (in MOD we call this a cascading set split, but it could include any combination of force-fail-to-syncs and set split scenarios). This is dependent on the manifestation of the non-universal I/O events caused by the particular sMIA diode failuremode. For the STS-124 case we were left with two GPCs in the redundant set when the FA2 bypassed, but this is not the worst-case manifestation.
• This leaves the BFS Standalone and susceptible to the BFS Blackzones. If the BFS is not Engaged quickly (less than ~ 10 seconds), it is susceptible to the LOCV Blackzone scenarios.
• It is more difficult to diagnose set splits than the total loss of PASS via set quit or halt scenarios, and staying under the 10 second limit is viable but operationally challenging.
Failure History (cont)
NASASpace
flight.
com