EG2401 T308 Project7 Final Report

NOUVELLES CONSULTANTS

Software and Product

Safety

EG2401 Project 7 Final Report

Muhammad Ahad U084075R

Muhammad Nadzri Hussain U096036E

Ruben S/O Sukumar U084705E

Farhan Bin Mohamad U090387B

3/10/2011

An analysis of the technical, security and ethical issues involved in the Ariane 501 Rocket and Patriot

Missile Defence System failures that were caused by software malfunction.

EG2401 Project 7 Final Report Tutorial 308

Page 1 of 17

1.1 - Ariane 501 case

1.1.1 Case overview

On June 4, 1996, the maiden flight of the Ariane 501 launcher ended in a

catastrophic failure. About 40 s after initiation of the flight sequence, at an

altitude of 2700 m, the launcher veered off its flight path, broke up, and

exploded. The accident and the inquiry report claimed that there was a

complete loss of guidance and attitude information, 37 s after the start of the

main engine ignition sequence and 30 seconds after lift-off.

1.1.2 Immediate Follow-Up Action

Immediately after such a disaster, the engineers from the Ariane 5 project

teams of Centre National d’Etudes Spatiales (CNES) and Industry

immediately started to investigate the failure. Also the Director General of

European Space Agency (ESA) and the Chairman of CNES set up an

independent Inquiry Board to examine the cause of the failure. Investigations

were carried out on the causes of the failure, the systems supposed to be

responsible, and any failures of similar nature in similar systems, and events

that possibly could be linked to the accident.

1.2 - Factual Issues

1.2.1 General Failure Analysis

Consequently, we will we be analysing the number of systems that failed,

the causes and conceptual issues regarding the causes of failure. The

investigation of its flight from the point of take of has to be thoroughly

assessed. The sequence of technical data that was traced from the flight data

records and the events that caused the failure are summarised in the table

below.

System Event

SRI and failure of the back-up Inertial Reference

Active SRI followed by the failure of the active Inertial Reference System

Solid

boosters

swivelling into the extreme position of the nozzles of the two solid

boosters


Page 2 of 17

Vulcain

engine

failure caused launcher to veer abruptly

Rupture of

links rupture of the links between the solid boosters and the core stage

Self-

destruction

of the

launcher

due to the rupture of links

Table 1: Information extracted from (LIONS, 1996)

Hence, considering the sequence of events it was evident the root of the

problem or the initial error that triggered the sequence of failures was the

flight control system which is characterised by the SRI as it failed to function

accordingly.

1.2.2 Function of SRI

The function of the SRI was primarily for the purpose of measuring the

launcher characteristics and its movements in space. Having an independent

control system (internal computer) which in turns calculates angles and

velocities based on information from a "strap-down" inertial platform, with

laser gyroscopes and accelerometers. The data from the SRI to the On-Board

Computer (OBC) executes the flight program and controls the nozzles of the

solid boosters and the Vulcain cryogenic engine, via servo-valves and

hydraulic actuators. (LIONS, 1996). Two SRIs work in parallel and there are

also two OBC to ensure reliability and at any time only one is active and the

other is in standby. If failure is detected in a single system, it automatically

switches to the other unit only if it is functioning properly.

1.2.3 Why SRI Failed?

The software design which was used in SRI in Ariane 4 (An earlier version

of the launcher) was reused in the SRI design in Ariane 5. However after the

investigation it was discovered that the software error occurred in the active

SRI and the main control system did not switch to the other SRI due to the

similar software problem. The internal SRI software error was caused during


Page 3 of 17

execution of a data conversion from 64-bit floating point to 16-bit signed

integer value. The floating point number which was converted had a value

greater than what could be represented by a 16-bit signed integer. This caused

a command error which resulted in internal alignment function known as

horizontal bias (BH) which is related to the horizontal velocity sensed by the

platform. This resulted in the shutdown of both SRIs.

1.3 - Conceptual Issues

1.3.1 Culture of the Ariane program

The ethics behind the software design is extremely complex to deal with.

However the one of the cause could be the culture of the Ariane Program

which only addressed the random software failures which could be efficiently

dealt with by back-up systems. The design of the software was such where the

2 SRIs would shut down when an exception was raised is an unnecessary

function in the software. On an ethical context, this type of failure should have

been simulated even before failure. The process creating such complicated

software involved addressing problems such as possibility due to failures.

However according to the inquiry report a bias nature was adopted in

solving of random failures. Since the supplier of the SRI was only following

specifications or design requirement provided by the host company, which

stipulated that in the event of any detected exception the processor was to be

stopped. However the exception which occurred was not due to random

failure but a design error. Even though the exception was detected it was

inappropriately handled because the view had been taken that software should

be considered correct until it is shown to be at fault. The board has reason to

believe that this view is also accepted in other areas of Ariane 5 software

design. After complete investigation the inquiry board was in favour of the

opposite view, that software should be assumed to be faulty until applying the

currently accepted best practice methods can demonstrate that it is correct.

(Aviation Week and Space Technology, 1996)


Page 4 of 17

1.3.2 No proper testing done

This is a grey area that needs to be addressed. Years of testing were done,

but much emphasis was only hardware failures and problems. Due to the

critical nature of the software in space launchers, proper modelling of software

design is vital to prevent disasters. The extensive tests conducted during the

Ariane 5 development program did not include proper analysis and testing of

the inertial reference system or of the complete flight control system, which

could have detected the potential failure. Even though it is not possible to test

the SRI as a ''black box'' in the flight environment, unless one makes a

completely realistic flight test, it is still possible to do ground testing by using

simulated accelerometer signals in accordance with predicted flight

parameters, while also using a turntable to simulate launcher angular

movements. Such a simple and accurate test been could performed by the

supplier or as part of the acceptance test. This would have exposed the failure

and the problem would have been rectified. (Aviation Week and Space

Technology, 1996)

1.3.3 Adequate Software Review

From the inquiry report, it is evident that no proper software review was

done prior to launch or hardware simulations. No stringent rules or restrictions

were in place to ensure that software was up to standard and met requirements

for further testing. For such space missions one should note that thousands of

functions can be correctly performed and one mistake can be mission

catastrophic. However mistakes can be prevented by oversight, test, and

independent analysis. A proper software review was much needed to avert

such a failure.

1.3.4 Management and Organizational Factors

The coordination between supplier of SRI and Ariane 501 team and also

among Ariane 501 team has to be questioned. Furthermore, the Ariane 501

accident report is almost totally silent about organizational structure problems

as it does not describe the allocation of responsibility and authority for safety

nor does it mention any organizational or management factors that may have

influenced the accident. Proper procedure manuals were absent and problems


Page 5 of 17

that surfaced might have been ignored by senior aviation specialists. The

reason behind this lack of communication remains a mystery and is hard to

prove or totally dismiss that such problem existed. From the examination of

accident data, there were signs the initial problems were not properly dealt

with by the Ariane 501 team.

Also, a more transparent organization of the cooperation among partners in

the Ariane 5 would have averted the disaster. With excellent cooperation

among team players, together with clear cut authority and responsibility, is

vital to achieve system coherence. In most similar magnitude disasters,

inadequate transition from development to operations was the key reasons

cited for failure. Engineering management has at times sometimes has a

tendency to focus on development and to put less effort into planning the

operational phase.

1.4 - Ethical Issues

The ethical issues for this case have to be carefully examined to identify mistakes,

irregularities, weaknesses in management, development and operation problems and

the overall co-ordination. Issues will be examined with due regards to software failure

and software ethics. The cost of development of Ariane 501 which was around $7

billion and the total lost was around $370 billion US dollars and this makes the

financial lost is extremely drastic. Hence problems will be addressed form a ethical

point of view.

1.4.1 Utilitarianism

From rough cost benefit analysis, it should be noted that the cost of

developing proper software involves around 2 years, involving a lot of team

players which translate to man ours, and financially costing around billions.

The benefits here are not short term, but considered to be long term.

Reputation is also at stake here and hence it is evident that cost-benefit

analysis cannot be reduced to just dollars here. So when the engineers who

designed the software by implementing the same software of Ariane 4 on the

Ariane 5 without evaluating all possible aspects or scenarios that will result


Page 6 of 17

due to this. Under the Rule Utilitarianism approach, certain good practices

were broken and this resulted in a bad design of the software.

1.4.2 Duty ethics

One might come to the conclusion that duty ethics is not of mush

importance as there was no fatality. However the software designers had a

professional duty to fulfil. They had a task of producing complex software for

the engineers who were working with the hardware design. Since the inquiry

report had clearly stated that proper testing of the software was not done

efficiently to determine potential problems, the duties of the software

engineers are seriously flawed. Apart from the software engineers, the co-

ordination of key Ariane 501 team players was lacking and everyone had a

responsibility to work together to ensure that the launch was successful.

Communication breakdown together with failure to exercise proper

responsibilities have illustrated the importance of duty ethics. The duties of the

higher management come into serious question. They should have initiated

proper software reviews for the Ariane 501which could have detected the error

at the start. Every engineer in the 501 program has the right and duty ensure a

successful launch.

1.4.3 Virtue Ethics

Certain actions of engineers and professionals do not reflect good character

traits. Loyalty to the profession is an ethical code all engineers should follow.

Apart from this, loyalty to employers is also a character trait that successful

engineers should have. Such qualities were absent from the Ariane 501

industry partners and engineers. Negligence and careless nature of the

software design team was reflected in the post- accident analysis by experts.

Such a negligent attitude has cost billions and has left a huge dent in the

Ariane reputation.

1.4.4 Self –Realization Ethics

The aspect Ethical egoism is also an issue here. The idea of long-term well-

being rather than a narrow, short-sighted pursuit of immediate success or

pleasure have to be examined carefully. From the whole process from research


Page 7 of 17

to development, testing and launching, it can be concluded that the search or

mind-set for immediate success was present among top management. Hence

careless or negligent attitude might have resulted from this trait. Such a rush to

ensure success has caused such a horrendous catastrophic failure.

1.5 - Recommendations and current practices

Based on the above analysis and the inquiry report following recommendations would

have helped to minimize the possibility of the Ariane 501 disaster.

The alignment function of the inertial reference system should be switched off as soon

as lift-off occurs. It should also be ensured that no un-necessary software runs during

flight. The system should be thoroughly tested by using as much real data as possible.

Also, flight simulations should be conducted before any mission. All sensors should be

engaged and send their best captured data. Thus, no sensor should be allowed to stop

sending its best effort data. Furthermore, specific software qualification reviews should

be conducted. Industrial Architects or whoever is in charge should participate in such

reviews and report on complete system testing together with equipment analysis.

Additionally, all flight software (including the embedded software) should be

comprehensively reviewed.

o This includes checking the assumptions (explicit and implicit) in the software code

against the constraints on equipment usage.

o The range of values taken by variables in the software should be realistic.

o Potential problems in the on-board computer software should be proposed by the

project team and then reviewed by a group of external experts.

Also, whenever it is technically possible, exceptions to tasks should be confined and

backup capabilities should be devised. More data should be provided to the telemetry

when a component fails, so that it would be less essential to recover the equipment. The

definition of critical components should be reconsidered, by taking failures of software

origin into account, especially single point failures.

External parties should also be consulted while reviewing specifications, code and

justification documents. Such parties should not be involved in the project and should


Page 8 of 17

consider the substance of arguments rather than check that verifications have been carried

out. The trajectory data should also be included in specifications and test requirements.

The test coverage of existing equipment should be reviewed and extensions should be

made where necessary. The justification documents should be regarded as important as

the code and techniques for ensuring consistency between code and its justifications

should be consistent.

A team should be set up which would prepare the procedure for qualifying software,

propose stricter rules for confirming such qualifications and ensure that the specification,

verification and testing of the software are of a consistently high quality in the Ariane 5

programme. This should also include external experts.

Lastly, there should be a more transparent organization of the cooperation among the

Ariane 5 programme partners. Engineers should collaborate very closely and there should

be clearly defined roles of authority and responsibility to ensure system coherence.

1.6 – Conclusion

The Ariane 501 failure is a multi-faceted example of how software errors, lack of

adequate testing, maintenance and coordination can lead to big disasters, resulting in a

multi-million dollar loss.

There are several issues involved in the failure. The usage of the old specifications

from Ariane 4 was not compatible with Ariane 5 (since its path was considerably

different). The testing performed was quite inadequate and did not take into account many

factors involved. The software-equipment compatibility was also not tested to as far an

extent as possible. The various parties involved in the project did not have a clear,

transparent communication interface. Thus, all of these issues led to the failure of Ariane

501. We saw that eagerness to achieve success can often lead to short-sightedness and

make organizations overlook essential safety precautions and thorough testing which can

result in disasters like this. Hence, it is important to adopt a strategy of comprehensive

error testing, maintenance, coordination between teams involved in the project and

ensuring that different system components are highly compatible. In order to achieve

these objectives, it is also very helpful to consult third party or non-partial experts that

can review and further check the system for any discrepancies.


Page 9 of 17

2.1 - Patriot MDS case

2.1.1 Case overview

In 1991, a war in the Middle East broke out commonly known as the Gulf

War. The war was led the Americans and Britain against Iraq, the army of

Saddam Hussein. Operation Desert Storm was the name given for the battle

occurred in the Gulf War mainly because of the operations and military

participation.

In February 25th

, 1991, a Patriot missile defence system, failed to track and

intercept an incoming Scud missile during the Operation Desert Storm in

Dhahran, Saudi Arabia. This failure claimed 28 American soldiers in an Army

Barrack and is considered to be one of the worst software failures in history.

2.2 - Factual Issues

When the Patriot missile defence system was first invented, it was originally meant

to operate in Europe. It was used as a defensive measure against the Soviet’s cruise

missile and medium to high altitude aircraft missiles, which were able to travel up to

twice the speed of sound. The Patriot system provides good mobility so as to prevent

contact. Because of its inconsistent location, it can only be operated for a few hours

before moving on to its next location.

In Operation Desert Storm, the Patriot missile system was deployed for the very

first time against the Scud missiles. The scud missiles, they can reach speeds of up to

five times the speed of sound. Scud missiles were new to the Army and any data

regarding the missile were very scarce. The Army had to rely a lot from external

sources especially from the intelligence agency. Without proper knowledge or data of

the Scud missile would make the Patriot missile system irrelevant. Therefore it was

very important that the Army had to understand how tracking and intercepting the

Scud missiles could be achieved effectively.

According to reports “The Patriot Battery at Dhahran failed to track and intercept

an incoming Scud missile because of a software problem in the system’s weapons

control computer”. This resulted in an inaccuracy in the calculation that worsened as


Page 10 of 17

the system operated longer. Based on findings, the battery has been operating for 100

hours. And by then the inaccuracy was so serious enough to cause the system to look

in the wrong place for the incoming Scud missile.

The irony was that the Patriot defence system had never been used before to

defend against Scud missile nor was it able to operate in long operating hours.

Reports indicated that Army officials received a data from Israeli two weeks before

the incident. The data indicated some loss in accuracy after the system had been

running for 8 hours. However, modifications were made to the system to improve the

software but it did not reach Dhahran until February 26, 1991. The day after the Scud

missile hit the Army Barrack.

2.3 - Conceptual Issues

The weapons control computer is actually the most important lifeline for the

Patriot missile system. However, the Patriot missile system that was used in Operation

Desert Storm was designed back in 1970s. This shows a very huge limitation in

precision tracking and intercepting, which is the main purpose of the missile.

To understand how radar works, it sends out electronic pulses in the air that is

surrounding it. These electronic pulses when hit a target would bounce back to the

radar thus giving information such as distance and location. It is similar to the

working principle of the Patriot’s missile system. An additional feature to the system

is the ability to select specific missile types that is required for the operator. And for

the case of the Patriots missile system, Scud missiles were selected as the target of

choice. The system's range-gate algorithm is the important factor in the accuracy of

the missile’s position.

When a flying object has been detected by the Patriot’s radar, the range gate area

would calculate only the area that the object is flying and filters out anything that’s

outside the area. Only when the object is in the area and calculated, then can it be

confirmed that the object is a scud missile. The range gate area requires that the

missile be exactly in the centre. And it follows the same throughout where the system

would next look out for it.


Page 11 of 17

Data came running through on 11 February 1991, that there was a 20% shift in the

radar range gate area after the system has been running for 8 hours consecutively.

This shift meant that the target was not in the centre of the range gate area and it

proves to be a hazard for the system. A shift in target would mean that the radar could

lose track of the missile and no longer intercept. As stated in the incident report

“Patriot Project Office officials said that the Patriot system would not track a Scud

when there is a range gate shift of 50% or more”. Based on simple calculations a 50%

shift in the range gate area would approximate to about 20 hours of continuous

operations. Basically in conclusion, running the Patriot system for more than 20 hours

would spells danger as the inaccuracy becomes too great. Thus, this results the radar

to not be able to detect the scud missile. For the case on the 25 February 1991, the

Patriot radar system has been running for 100 hours consecutively and has shown how

fatal and deadly the system could be.

2.4 - Ethical Issues

Assumption made by the army officials that the Patriots were not running their

systems for more than 8 hours at a time was one of the key factors that caused the

disaster. However, when the Israel data was analysed and loss in targeting accuracy

was detected, the officials made software changes which rectify the inaccurate time

calculation. The change made it possible for the system to run longer and was part of

the modified software version that was released on February 16, 1991

On February 21, 1991, the Patriot users received a message from the Patriot

Project Office. It was stated that running the system for a ‘long’ period would result in

a shift in the range gate, which would cause in the target being offset. It was also

stated in the message that software update was being sent that would amend the

system targeting. On the other hand, the message did not specifically narrate the

definition of very long run times. This was because the army officials assumed that

the users would not run the system batteries continuously for such a long period of

time which would cause the error to emerge. This leads to the army official not giving

sufficient information as they though it was not required.


Page 12 of 17

Unfortunately after the Israel data was received, immediate actions was not taken

by the Army officials to determine the period that the Patriot missile could operate

before the inaccurate time calculation would cause the system to be ineffective. This

caused the modifications being sent out to the system later than expected.

Ethical issue is the major root of this disaster as it leads to a lot of other problems.

Software systems can be more complicated therefore it needs very thorough review

process and testing. The engineers failed to discover all bugs as although it is difficult

to test all combinations of inputs or variable values. They also did not ensure that the

rectification was done immediately. The complacency that the system will not run for

more than 8 hours was also one of the ethical issues. This shows that software

engineering principles may not be adhered properly.

2.5 - Analysis of the Ethical Issues

From this incident, a few ethical issues can be discussed. Although the main cause

is in the numerical error, there were a lot of events that led up to the disaster. This was

hugely contributed by ethical issues. As a result, it caused a catastrophic failure

involving loss of human life.

Inconsistency was one of the primary factors of this mishap. When the error was

discovered at Raytheon, the amendment was only made to some areas of the codes.

This caused the inconsistency in the codes as they were not operating in the same

function. The updated codes were running with a faster and more precise function.

This program was developed fifteen years ago in assembly language. With updates

and new codes added, it made the system tough to operate and maintain. Therefore the

defect was not identified before the incident.

In addition, complacency was another cause of the disaster. The army officials did

not bother to investigate the number of hours that the system will be running. By

assuming that the systems will not run for a very long time continuously caused the

message received without any detailed specification of when the system will incur

error.


Page 13 of 17

Another factor that contributed to the software engineering ethical issues is

maintainability. It is very important to maintain an old system. In 1986, when the

Patriot Missile System was modified to enhance missile tracking, the developers

chose to recode the software in a high-level language. This would improve the system

performance and maintainability. The fifteen year old assembly code that constantly

patched was too complicated for the programmers to understand. This caused them

the inability to do a better job when dealing with the system. The checks for a critical

safety system should be done comprehensively. It should have been detected through

vigorous debugging and unit testing, though it can be quite intricate and expensive.

The problem could also be identified if retesting is done after every update or

patching.

Reaction time taken by the army officials to rectify the software problems was not

fast enough. The officials should have reacted with much prompt and effectiveness.

This delay in the response leads to the disaster as the message was not received on

time. An ethically responsible software engineers would have taken the matter

seriously and reacted accordingly to prevent this mishap.

In addition to the bad timing of the bug fix’s late arrival, it causes the loss of life in

Dhahran. Twenty-eight soldiers lost their lives while serving their country. Had better

care been taken to conform to ethical standards, their deaths might have been avoided.

2.6– Recommendations

Ethical Responsibilities should be instilled in all individuals dealing with the

system software. They should avoid falsifying test results or covering up known or

suspected problems. Leaving potentially serious bug in a product and violating safety

standards. When adapting an older software system to a new user, the new user should

understand the whole system before operating on them.

There must be sufficient analysis of potential failure modes and consequences. To

design and implement appropriate safety and robustness features. They should have

reacted aggressively to reports and warnings after the possible problems.

Engineers and management should not underestimate the complexity of the

software. Poor software engineering techniques should not be use given that there was


Page 14 of 17

little to no documentation available regarding the developments of the software. The

management should react faster to communicate the problems with other users. Most

importantly protect against error and not error discovery.

In making software engineering a beneficial and respected profession, software

engineers shall commit to making design, analysis, testing, specification, maintenance

and development. Software engineers shall follow these following Eight Principles, in

accordance to their commitment to the welfare, health and safety of the public:

1. PUBLIC - Software engineers shall act consistently with the public

interest.

2. CLIENT AND EMPLOYER - Software engineers shall act in a manner

that is in the best interests of their client and employer consistent with the

public interest.

3. PRODUCT - Software engineers shall ensure that their products and

related modifications meet the highest professional standards possible.

4. JUDGMENT - Software engineers shall maintain integrity and

independence in their professional judgment.

5. MANAGEMENT - Software engineering managers and leaders shall

subscribe to and promote an ethical approach to the management of

software development and maintenance.

6. PROFESSION - Software engineers shall advance the integrity and

reputation of the profession consistent with the public interest.

7. COLLEAGUES - Software engineers shall be fair to and supportive of

their colleagues.

8. SELF - Software engineers shall participate in lifelong learning regarding

the practice of their profession and shall promote an ethical approach to

the practice of the profession.

The role and responsibility of a software Engineer is to act professionally and

ethically by understanding and appreciating risk factors. They are to ensure proper

attention is paid to these principles


Page 15 of 17

2.7 - Current Practices

Current situation for the Patriot Missile would be to restart the system for 2 to 4

hours. Turning off and on the system helps the system to refresh itself and reinitialize

the computer time back to zero. To reboot the system it would take only less than a

minute.

Another immediate action taken by the Army would be to provide advanced

training for the Patriots maintenance operator. Raytheon has developed several

training to take the highly experienced ones to a much higher level of maintenance

operator. Every update in the software or upgrades is being taught to the maintenance

operators. Thus in time, there would be many highly trained and skilled soldiers.

Lastly though still under prototype, conventional radar has been replaced by an x-

ray radar system. Unlike the conventional radar, it do not use electrical pulse or radar

waves to detect a missile but instead uses x-rays that is much more superior. Hence,

the name x-rays radar system, Therac-12.

2.8 - Conclusion

In conclusion, we realized how much people these days rely on such software or

computers to help us protect ourselves or even help to save lives. The responsibilities

placed on such small software or computer is so great, thus needed a great care on the

software or computers. It is important to do regular maintenance and checks on the

system that everything is running fine and nothing peculiar in the software that can

result in the malfunction or the ability to not function properly.

Due to the reason that software is very prone to bugs and viruses, it is also

important to debug the software and do modifications or updates on the system. For

example, how Apple or Microsoft conducts regular updates on their software, and if

there were any bugs or viruses they would send updates or upgrades for downloading.


Page 16 of 17

References

Bibliography (n.d.). Retrieved from http://www.engineering.uiowa.edu/~ece_036/Lecture/Ethics.pdf

Arnold, D. N. (1996, September 24). Two disasters caused by computer arithmetic errors.

Aviation Week and Space Technology. (1996, September 12). Ariane 5 Report Details Software

Design Errors.

LIONS, P. J. (1996, July 19). Flight 501 Failure. Retrieved March 1, 2011, from

http://www.di.unito.it/~damiani/ariane5rep.html

Lum, A. (n.d.). Patriot Missile Software Problem. Retrieved from

http://sydney.edu.au/engineering/it/~alum/patriot_bug.html

Math News. (2003, October 17). New anti-missile system declared worst engineering disaster EVER .

Retrieved from http://www.mathnews.uwaterloo.ca/Issues/mn9303/arienne.php

Raytheon Company. (n.d.). Raytheon. Retrieved from "Gulf War" :

http://www.pbs.org/wgbh/pages/frontline/gulf/weapons/raytheontext.html

The Importance of Ethics in Mission and Safety Critical Software Engineering. (n.d.). Retrieved from

http://cseserv.engr.scu.edu/StudentWebPages/uletran/uletran_FinalPaper.htm

Documents

EG2401 T308 Project7 Final Report