Embed Size (px)
Citation preview
Revised Syllabus 2007
1. Concept of Socio-Technical systems:
2. Why Systems Fail
3. Why Products Fail in the Market
4. Quality Function Deployment
5. Managing Risk in Software Development
Syllabus
Monday 19th February10-11 N123 Introduction and outline of assessment11-1 N123 Sociotechnical systems 2-5 N123 Technology Deployment: managing change and introducing technology
Tuesday 20th February10-11 N123 Participative design11 – 1 N123 Varieties of technology failure 2-5 N123 Technology diffusion: innovation and market penetration
Wednesday 21st February
10-1 N123 Quality Function Deployment2-5 N123 Managing Risk in Product Development
Assessment
Assignment (60%) Case study of specific product success in market and guidelines
to minimise risk of future failure.
Assignment 2 (40%) Comparison, using Quality Function Deployment, of new
technologies
Aims
The Aims of this module will be:
to consider why technology fails;
to understand how diffusion of technology is affected by market and cultural forces;
to introduce the concept of risk in design;
to investigate methods for managing the introduction of technology to organisation;
Objectives
After completing this module, students should be able to: define factors which can impair a technology’s
acceptance; plan product development to address and
minimise risk using QFD; plan the introduction of new technology to an
organisation
7
System Engineering (ISO 15288)
Special process
Tailoring
Acquisition SupplyAgreement processes
Project processes
Project planning
Project control
Decision-making
Information management
Risk management
Configuration management
Project assessment
Enterprise processes
Enterprise environment management
Investment management
System life cycle processes
management
Resource management
Quality management
Technical processes
Disposal
Stakeholder requirements
definition
Requirements analysis
Architectural design
Implementation
Integration
Verification Transition Validation Operation Maintenance
Socio-technical Systems
Sociotechnical Systems
The defining characteristics of systems thinking are the twin notions of “a complex whole” formed from a “set of connected things or parts” (Allen, 1984).
“regularly interacting or interdependent group of items forming a unified whole”.
Rational or Industrial Age ‘Closed’ Systems
typically concerned with the attainment of a relatively specific goal
well specified criteria for deciding on optimum means to ends
“a relatively high degree of formalization”
“conscious and deliberate” corporation among participants (i.e. the organisation is ‘designed’).
Post Industrial, Sociotechnical ‘Open’ System
“may spontaneously re-organise towards states of greater heterogeneity and complexity
achieve a ‘steady state’ at a level where they can still do work.
grow by processes of internal elaboration.
achieve a quasi-stationary equilibrium in which the enterprise as a whole remains constant, with a continuous throughput, despite a considerable range of external changes.”
Sociotechnical Systems
Socio and technical elements both become system elements that have boundaries through which there is a transaction or exchange of information.
A greater part of the ‘world’ is admitted into the analysis but the trade off is that the linkages between system elements cannot be known or specified to the same degree of ‘tightness’.
Sociotechnical Systems
Not just structural and technological aspects of a work domain (in which the nature of the interconnection is knowable and publicly observable) but also psychology and sociology (in which interactions and linkages are more probabilistic in nature and difficult to measure).
Thus a sociotechnical system often has the properties of a network rather than an object.
Miners
Long wall method of mining Durham pit: new cutting machine, c1940 Original design
Hewer + mate + trammer Paid by weight to surface
Machine-design Shift 1: cutting (paid by hole, yard
or day) Shift 2: ripping (paid by yard or
area) Shift 3: filling (paid by weight)
Redesign Multifunction teams
Sociotechnical TheoryResponsible Autonomy
“the strong need in [military personnel] for a role in a small primary group”.
Immediacy of interaction and proximity of trusted team members
“under [operational] conditions there is no possibility of continuous supervision, in the factory sense, from any individual external to the primary group”
“Groups of this kind were free to set their own targets” (Trist & Bamforth, 1951, p. 7)
Sociotechnical TheoryAdaptability
“aspiration levels with respect to [orders] could be adjusted to the [characteristics] of the individuals involved” and to local conditions
Simple organisations doing complex things Problems do not become magnified through a
larger social space because there is a far lesser extent of task interdependence (Trist & Bamforth, 1951, p.21).
Sociotechnical TheoryMeaningfulness of Tasks
“…placing responsibility for the complete [military objective] squarely on the shoulders of a single, small, face-to-face group which experiences the entire cycle of operations within the compass of its membership.”
“for each participant the task has total significance and dynamic closure” (Trist & Bamforth, 1951, p. 6).
“conditions under which those concerned can complete a job in one place at one time, i.e., the situation of the face-to-face, or singular group” (Trist & Bamforth, 1951, p. 14).
Job DesignTechnologyEquipment
LayoutMethodsMaterials
InformationArchitecture
Management StructureAuthorityRecruitmentSupervisionEvaluationCompensationPromotion
Technological resourcesTechnical tasks
Human resourcesOrganisational tasks
JOB CONTENT AND STRUCTURE
The Case of the DVLC
Driver and Vehicle Licensing Centre “…each operator had a very narrow, unskilled job to carry
out: opening post; examining contents; editing; codings; applying batch numbers; microfilming; key-punching etc.”
Original Specification: £146 million to implement and 4,000 employees
Implementation: £350 million and 7,900 employees
Source: Heller, 1989, In Bamber and Lansbury, New Technology
Volvo
Novel factory design
Wagon-based work-flow under computer control
Semi-autonomous working groups
Sociotechnical TheoryResponsible Autonomy
Immediacy of interaction and proximity of trusted team members
“under [operational] conditions there is no possibility of continuous supervision, in the factory sense, from any individual external to the primary group”
“Groups of this kind were free to set their own targets” (Trist & Bamforth, 1951, p. 7)
Sociotechnical TheoryAdaptability
Simple organisations doing complex things
Problems do not become magnified through a larger social space because there is a far lesser extent of task interdependence (Trist & Bamforth, 1951, p.21).
Sociotechnical TheoryMeaningfulness of Tasks
“for each participant the task has total significance and dynamic closure” (Trist & Bamforth, 1951, p. 6).
“conditions under which those concerned can complete a job in one place at one time, i.e., the situation of the face-to-face, or singular group” (Trist & Bamforth, 1951, p. 14).
Job DesignTechnologyEquipment
LayoutMethodsMaterials
InformationArchitecture
Management StructureAuthorityRecruitmentSupervisionEvaluationCompensationPromotion
Technological resourcesTechnical tasks
Human resourcesOrganisational tasks
JOB CONTENT AND STRUCTURE
Socio-technical Systems
Semi-autonomous working groups
Joint Optimisation of social and technical Coal mining in Durham
machine-driven shifts vs. multiskilled teams Volvo factories
Novel factory design and semi-autonomous working group Redesign of DVLC
Introduction of 18-person teams and redesign of jobs
Sociotechnical Design Principles1. Systemic. “…all aspects of organisational
functioning are interrelated”
2. Open System “…continuous adaptation to requirements flowing from environments”
3. Joint Optimization. The principle that socio and technical elements of an organisation should be jointly considered and maximised.
4. Organisational Uniqueness. “…Structure of the organisation…suits the specific individual organisation’s situation” (relates back to adaptation above).
Source: Cherns, 1978, Human Relations, 40
Sociotechnical Design Principles
5. Organisational Philosophy. The design of structures and roles is “congruent with agreed organisational values” (In other words, not a ‘bolt-on’ solution but pervasive and ubiquitous).
6. Quality of Working Life. “…integrity, values, and needs of individual members are reflected in the roles, structure, operations, and rewards of the organisation.”
7. Comprehensive Roles of Individuals or Groups. The content of work and the people used to carry it out (and their organisation into teams or groups) should reflect the principles of ‘meaningful’ and ‘whole tasks’.
8. Self-Maintaining Social Systems. “…social systems are such that organisational units can carry on without external coercion…i.e. they are to become self-regulating”.
Source: Cherns, 1978, Human Relations, 40
Sociotechnical Design Principles
9. Flat Structure. Although somewhat contrary to historical notions of military hierarchy, the attribute of a sociotechnical system (one that is jointly optimised) is that there are “fewer organisational layers or levels”.
10. Participation. “…democratization of the work place” with individuals able to contribute to problem solving and governance.
11. Minimal Status Differentials. This attribute seems to run counter to military thinking in terms of there being “minimal differences in privileges and status”, but on closer inspection it can be noted that any differences which are “unrelated to role and organisational needs” are regarded as divergent from a sociotechnical ideal.
Source: Cherns, 1978, Human Relations, 40
Sociotechnical Design Principles
12. Make Large Small. “Organisational and physical structures provide both smaller, more intimate organisational boundaries and a feeling of smaller physical environment for individuals or groups”.
13. Organisational Design Process. “…components of the organisation evolve in a participative, iterative manner, only partially determined by advance planning”
14. Minimal Critical Specification…This principle is (tacitly or otherwise) at the heart of Effects Based Operations. In organisational design terms, “…designers specify (design or select) the crucial relationships, functions, and controls, leaving to role-holders the evolutionary development of the remainder.”
Source: Cherns, 1978, Human Relations, 40
Exercise
A simple order processing activity Receive mail Open mail Retrieve payment slip Retrieve cheque Record slip details Record cheque details Reconcile payment Create reply Mail reply
Exercise
Order the tasks according to: Stages,
e.g., input – process – output Technology,
e.g., paper – computer Objects,
e.g., mail – slip – cheque Job role,
e.g., mailclerk – computer operator – accountant - supervisor
Technology Deployment:
managing changeandintroducing technology
Inter-relationships
People
Technology Organisation
Tasks
Source: Leavitt and Bahrami, 1988, Managerial Psychology
Approaches
Phases
Review
Objectives
Design
Implement
Muddle through
Awareness
‘quick fix’
External
DebugStaffing
Technocentric
AutomateReduce labourCentralisationImprove workClean and final
DebugTraining
Human-centred
Retain people
DecentraliseEnrich workParticipation
ProcessEmpowerment
Source: Blackler and Brown, 1987, in Oborne IT and People
Strategies
Big Bang
Parallel running
Phased introduction
Trials and Dissemination
Incremental evolution
Revolution
Evolution
Adaptation
Assimilation
ETHICS
Effective Technical and Human Implementation of Computer-based Systems
Specify mission Describe work Work morale
Reorganise Organisation Objectives Specifywork options change
Technology Training Redesign ImplementSelection of staff work change
Evaluate
Source: Mumford, 1986, ETHICS
Exercise
You own an electronics component store
There are 10 members of staff, few of whom are computer literate
Use ETHICS to propose a strategy for implementing a computer stock management system
Cost:Benefit Assessment
Technical systems specification
Organisational Description
User cost-benefit assessment
Organisational cost-benefit assessment
Socio-technical design
Participative Design
What is Participative Design? Potential users:
Consulted in design Involved in design Doing the design
Defining users
Why isn’t participative design more widely used? Do we know who the ‘users’ are? Do we know how the ‘users’ will participate? Does it lead to more cost effective use of systems? Does it lead to faster design process? Does it lead to better user attitudes? Does it lead to better fit between requirements and
system delivered?
Approaches
Consulted in design Requirements capture and specification User needs analysis
Involved in design Prototypes and trials Critique and evaluation of concepts
Doing the design Defining requirements and functionality Proposing architecture
User Participation Techniques
Eason, 1989, in Knight, Participation in Systems Development, London: Kogan Page
Design phase User participation techniques
1. Feasibility
1.1 analysis of socio-technical options
1.2 user cost:benefit assessments
1.4 design team composition and strategy
2. Requirements specification
2.1 organisational and task analysis
2.2 job design
2.3 specifying technical system criteria
2.4 user evaluation of prototypes
3. Selection & design of system
3.1 usability design and testing
3.2 pilots and trials
4. Implementation, support and evolution
4.1 implementation procedures
4.2 customisation
4.3 workplace design
4.4 user support and development
4.5 user audits of systems
Socio-technical Systems Design
Conceptualdesign
Architectureconstruction
Functionalrequirements
Allocation of function
Coding
Interface spec
Documentation Maintenance
Define user needs
Organisationaldesign
Personnelpolicy
Job design
Workstationdesign
Change process
Organisationalobjectives
Testing
ImplementationAnd support
Source: Eason, 1988, Job Design and Human Work
Varieties of technology failure
Failure…
Human error
Lack of desired outcomes
Lack of diffusion / market penetration
Human Error
70% - 90% of accidents in industry and aviation attributable to ‘human error’
Humans as significant source of technology failure?
Human Error
Failure of person or failure of design?
Lack of intelligence or lack of information?
Lack of ability or lack of guidance?
Types of human error
Humanactivity
Unintendedactions
Intendedactions
Slip
Lapse
Mistake
Failure of attentionIntrusionOmissionReversal
Misorder / Mistime step
Failure of memoryOmit stepLose place
Forget intentionWrong procedure
Rule breaking ViolationSabotage
Some Working Assumptions
People interpret technology in terms of their previous experiences
People interpret technology in terms of the ‘system image’
Errors arise from inadequate design of technology or jobs, as well as from external factors
Norman’s 7 Stages of Action
GoalIntention Evaluation
Plan of action Interpretation
Action Perception
Things in the World
Source: Norman, D.A., 1990, The Design of Everyday Things, New York: Basic Books
“Swiss-cheese”
model of human error
Action
Environment
Technology
Work practices
Management
‘path’ of error
Automating Work
Direct Automation Replacing human labour with technology
Indirect Automation Replacing some parts of human labour with
technology Supporting human labour Requiring support from human labour
Allocation of Function
Share Functions between system components
Humans are system components
Humans are allocated functions Because they are more efficient than technology Because they are more flexible than technology Because they are cheaper than technology Because it is hard to get technology to do the function
Human : Technology Efficiency?
Satisfactory
MachinePerformance
Unsatisfactory
Unsatisfactory Human Satisfactoryperformance
Um
UhPm
PmhPh
Umh
Increasing Automation and Human Activity
Contribution to mission
Manual Bounded Tele Supervised Adaptive autonomy operation autonomy autonomy
Monitoring Task planning Mission planning
Physical work
Irony of Automation
Technology applied to what can be automated Operators left with tasks that can’t be automated Unrelated set of tasks Operator ‘out of control loop’
Operators supposed to intervene Intervention requires operator in loop Intervention requires predictive model of system People very poor at monitoring for long periods
Out-of-loop Performance
Click button to select journey
Version One: Manual
Route Fuel Time187 4.2 gallons 2h 10m
FM97 3.2 gallons 2h 30m
486 3 gallons 3h 5m
Out-of-loop Performance
Click button to select journey
Version One: Automated
Route Fuel Time Probability187 4.2 gallons 2h 10m 25%
FM97 3.2 gallons 2h 30m 63%
486 3 gallons 3h 5m 12%
Effect on Human Performance
Decision Time (s)
40
30
20
10
1 2 3 4 5 6 trial
Automated
Manual
Source: Endsley and Kiris, 1990, Human Factors, 390
Deskilling
Removes opportunity to use learned skills Removes operator from control loop Reduces opportunity to make decisions Causes reliance on technology
The case of CNC
1. Motor skills2. Perceptual skills3. Conceptual skills4. Discretionary skills
Lathe operation: operator = all skills
CNC: machine = 1 + 4 Operator = 2 difficult, 3 not easy
Source: Hazelhurst et al., 1969, Occupational Psychology, 43
Job Design and CNC
Function
Programming
Editing
Setting
Operating
Charging
P SF MO P SF MO P SF MO P SF MO P SF MO
Source: Ullich et al., 1990, Int. Journal Industrial Ergonomics, 5
CNC+
Technology and Work
Technology is not ‘neutral’ Design ‘anticipates’ ways of working
Technology does not dictate work Installation and deployment influences use
Technology varies according to: Organisational context Management intentions Operator / user skill
Technology Diffusion:
innovation and market penetration
Lack of desired outcomes
Expected outcomes: Productivity etc
Direct vs Indirect automation
Why different trends?
Why did industrial sector benefit from technology?
Why is service sector so flat?
Why did banking/financial sector not benefit from technology?
Payback Shortfalls
Outdated operations that fail to keep pace with marketplace
Technology superimposes inappropriate work systems on organisation
Operations are excessively complex
Hackett, 1994, In Rhodes and Wield (eds) Implementing New Technologies, Blackwell
Factors
Socio-technical System Technology fit Organisational restructuring Capability of users Relationship with work
Case Studies
Missile Post 1959: 3000 letters sent in missile from submarine to
Florida naval base “Before man reaches the moon, mail will be delivered
within hours from New York to…Britain…by guided missiles.”
Pneumatic Post 1800s: Paris petit bleu Letters in metal cylinder and fired through pipes using air
pressure
http:www.islandnet.com/~ianc/dm/dm.html
Xerox
When the Haloid Company completed its development of a new type of plain-paper copying technology in 1959, it required capital to market the product. The company thus approached IBM. However, the consulting firm Arthur D. Little, hired by IBM to evaluate the potential of the new technology, saw no future in it and recommended against investing. Rejected, Haloid resorted to raising capital by selling extra stock. Their first copier was shipped in 1960. Eight years later Haloid, now renamed Xerox, achieved sales of more than $1 billion and revolutionised the way offices were run (Pool, 1997).
New Product Innovation
- only 2 out of 10 new products are a commercial success
- only 1 out of 8 hours of efforts results in technical success
- 46% of resources go to unsuccessful products
- 35% of products fail to sell
Booz-Allen & Hamilton Inc., 1982, New Product Management for the1980s, New YorkCrawford, 1979, Research Management Sept
Forms of Innovation
Product New product New materials New services
Market New applications New markets
Process Industrial Administrative Management
Exercise
Consider the following in terms of forms of innovation: The Mobile Telephone The Personal Digital Assistant The Digital Camera The Sony Walkman
What were the existing technologies? What were the innovations?
Design Steps
Landmark (c1%) Fundamentally new technology
Radical major (c9%) Substantial change to existing technology
Minor (c90%) Incremental Generation / version Improvement Detail Branding
Process or Patchwork?
Penny Farthing Racing bike
?
Developments
Different constraints… Women cyclists and skirts Elderly riders and mounting Safety from fall Comfort Stopping
…lead to different designs… …lead to different requirements
DaviesNPLPacketswitching
Early stages of the Internet…
1940 1965 1970
Bush‘Memex’
Engelbart‘augmentationmeans’
MIT / CTSS
Licklider‘Galactic computer Network’
BaranRAND‘hot PotatoRouting’
ARPA / ITO
NWG
BBN‘interface messageProcessor’
SRI‘national InformationCentre’
ARPANET
Technological Frames - 1965
Augmentation FrameTheory: cyberneticsTechniques: interactionUse: information
managementThought Leaders: Bush,
EngelbartSub-frames: microfilm,
radarLocus of activity: SRI
On-line Communities FrameTheory: ?Techniques: file sharingUse: effective use of
many computersThought Leaders: Licklider,
EngelbartSub-frames: SAGELocus of activity: MIT, SRI
Technological Frames - 1965
Networks Frame
Theory: information theory
Techniques: switching
Use: reliable comms
Thought
Leaders: Baran
Sub-frames: telephones
Locus of
activity: RAND, MIT
Electronic Services FrameTheory: ?Techniques: switchingUse: access to servicesThought Leaders: DaviesSub-frames: telephonesLocus of activity: NPL
Technological Frames
Bijker suggests that ‘technological frames’ shape the way a problem is approached Depend on current theory, methods, technologies Each ‘frame’ has its own set of assumptions and
approaches Success might depend on what degree of
inclusion to allow
Technology Adoption
Types of Adopter
Rates of Adoption of Innovation Rogers (1995)
Categories of Adopter: innovators early adopters early majority late majoritylaggards
0
50
100
150
200
250
300
S-Shaped curve over time
Bass Curves for Diffusion of Innovation Diffusion follows an S-shape curve
Such a curve can be described as:
Nt = Nt-1 + p (m – Nt-1) + q (m – Nt-1)Nt-1
mm = market potentialp =coefficient of external influence q = coefficient of internal influence
Standard Bass Curve[p =0.03, q = 0.38]
Org
anis
atio
ns h
avin
g ad
opte
d th
e in
nova
tion
Time after introduction of innovation
http://andorraweb.com/bass/index.php
Impact of ‘opinion leaders’
Scan page 113 figure 7.4, Valente…
Diffusion to mass market
Mobile Telephone Diffusion
0
10
20
30
40
50
60
70
1988 1992 1993 1995 1997 1998
% P
opul
atio
n
AustraliaChinaFinlandFranceGermanyJapanSwedenUKUSA
Source: EMC, 1999
1991 vs. 1996
Blue = internet coverage
http://som.csudh.edu/fac/lpress/articles/acmfwk/f1.htm
World Wide Internet Users January 2000
http://www.commerce.net/research/stats/wwstats.html
http://www.gvu.gatech.edu/user_surveys/survey-1998-10/graphs/general/q50.htm
Internet Diffusion Framework Pervasiveness, e.g., users per capita Geographic dispersion, e.g., access in cities vs.
rural areas Sectoral absorption. e.g., social sector using internet Connectivity infrastructure, e.g., telecommunications
coverage Organisational infrastructure, e.g., ISPs Sophistication of use, e.g., use for email vs. e-
commerce
Press et al., 1998, An Internet Diffusion Framework, Communications of the ACM, Vol. 41, No. 10, pp 21-26
Cuba 1998 (1)
Pervasiveness: c.100 users; even with UUCP email accounts, > 1/1,000 population, therefore Cuba is at the experimental level.
Geographic Dispersion: The only IP point offering network connectivity in Cuba is at CENIAI in Havana, although email connectivity in every province.
Sectoral Absorption: IP connectivity rare in health and government sectors, and nonexistent in education and commerce. Email is used in the health sector, more than 10% of the ministries have email accounts, and in education sector nationwide.
Cuba 1998 (2)
Connectivity Infrastructure: Cuba has an international IP link, but no domestic backbone.
Organizational Infrastructure: CENIAI and Teledatos provide connectivity to organizations with networks. Some coordination provided by the Inter-ministerial Commission for Networking.
Sophistication of Use: Email and information retrieval from email-driven servers have reached the conventional level in the health care and biotechnology communities.
Technology Acceptance Model Which innovations will be successfully
introduced into organisations?
System features influence perceived usefulness and ease of use
Based on Fishbein and Ajzen’s (1975) ‘Theory of Reasoned Action’
TRA schematic
Attitude
SubjectiveNorm
BehavioralIntention
Behavior
biei
cbkpfk
Theory of Reasoned Action
Overt behaviour Intention to behave (BI)
BI = w1 ((biei))+ w2 ((NBjMCj))
Or -
BI = w1(AB)+w2(SN)
AB – attitude towards that behaviourSN – subjective norm
Subjective Norms & Attitudes
SN = (NBjMCj)
NBj – perceived expectation of behaviour
MCj – motivation to comply
AB = (biei)
bi – expectation of outcome
ei – evaluation of outcome
For example…
A boss might believe that employees want her to use SMS (NBj) but she might feel that what they want is not important to (MCj)
She might believe that using SMS will make it easier for employees to contact you (bi) but not see this as desirable (ei).
Technology Acceptance Model
Davis (1982) Attitudes towards technology defined by: Perceived
usefulness (USEF) and perceived ease of use (EOU)
No role for subjective norm
30-40% of variance
TAM schematic
Attitude BehavioralIntention
TechnologyUsage
PerceivedUsefulness
PerceivedEase of Use
Taylor and Todd (1994)
Perceived ease of use; perceived usefulness; compatibility; peer influence; superiors influence; self efficacy; resource facilitating conditions; technology facilitating conditions
Results no better than TAM
Participation in implementation and /or design process seems to improve results
Example: TV home banking (1) 2000 viewers of BSkyB who could use Webtv to access bank account and perform transactions
Access to Webtv and to remote control Social and gender influences on tv viewing
and on use of remote control
Keeling et al., 2001, TV homebanking and the technology acceptance model, Interact01, Amsterdam: IOS, pp.84-91
TAM for TVHB (2)
NB ‘affective’ / Gender / Social component
AffectiveGender‘Social’
Task facilitation
Ease of use
Intention
Behaviour
Facilitatingconditions
TAM for TVHB (3)
Affect/Gender/social: 6%
Ease of use: 7%
Usefulness: 9%
Intention: 27%
Total: 49%
WAP set of specifications established in 1997 by WAP Forum [Ericcson, Nokia, Motorola, Unwired Planet: each had own protocol] Open, license-free? Scalable across transport options Related to relevant standards Extensible across new networks [http:www.inherent.com/articles/wap.cfm] Services in c. 1999
Forrester research institute proposed by 2004 1/3 of European mobile user (219 million) would use phones to access internet [http://technoaxis.com/main.wap.html]]
2.9% of 244 million mobile phones used to access WAP across Europe Millman – ‘WAP overhyped with respect to what can be delivered today’
http://www.financialdirector.co.uk/features/it/1111179 (why is wap failing to win over users?)
Technical
Phones on GSM 9.6kbps c. 5 x slower than standard modem, i.e., log on to email on 8 seconds on desktop, 75 s on WAP phone – circuit-switching for voice, so slow for lots of data
Long download time Paid per minute not quantity of data Would be better on packet-switching
020406080
100120140160
Year
Sale
s i
n M
illi
on
s
Mobiles sold
WAP enabled
[http://www.crt.dk/uk/staff/chm/wap/smsimode.PDF]
Interface
Nokia 7110 has 96 x 65 pixels Cf Ericsson MC218 640 x 240
Menu bar driven? Nielsen & Norman’s usability study (before > after)
Read world headlines – 1.3s > 1.1s Receive Guardian headlines – 0.9s > 0.8s Check local weather – 2.7s > 1.9s Read TV listings – 2.6s > 1.6s Asked if they’d use again 70% said no (if wait 3 years 20% still say no)
MetaGroup survey C 100 users – 90% stopped using WAP and used phone for voice
(ComputerWeekly.com) Usability by Design Ltd
[http:www.usability.uk.com/downloads/UsabilitybyDesign_Survey14.pdf]
Slowness (26%); lack of content (22%), cost (19%), limited display (12%) Success rate – 94% desktop browser vs. 36% for WAP
Innovation-Diffusion
Creativity
INNOVATIONDesign / product / manufacture
Radical / incrementalCharacteristics
LicensingManufactureMarketingDemand
DIFFUSIONAdoption by end users
Standing of producerTypes of adopterResistanceNeed
Market
Quality Function Deployment
QFD Process
Affinity Diagrams
Tree Diagrams
Matrix Diagrams
Quality Function Deployment
‘A system that translates user needs into the company’s specification in every step of the production process viz. marketing, development, production and sales and services’ American Supplier Institute, 1990
Worked Example
Source: Bergquist, K. & Abeysekara, J., 1996, Int. J. Ind. Erg., 18, 269-275“Quality Function Deployment (QFD) – a means for developing usable products”
Designing Safety Shoes:1. Customer / user needs – questionnaire survey:
Mobility Thermal comfortAppearance Ease of donningFit WeightDurability AdjustabilityAnti-slip Protection from hazards
Worked example
2. Ranking of Customer/user needs:Mobility [5] Thermal comfort [9]
Appearance [1] Ease of donning [4]Fit [10] Weight [8]Durability [3] Adjustability [2]Anti-slip [6] Protection from hazards
[7]
Worked example
3. Product characteristicsHow can needs be met? - Expert opinion:
centre of gravity leakproof
size permeability
bulkiness design of sole
insulation design of toecap
Building a ‘House of Quality’
‘Matrix’Needs x Product
‘Roof’:Product feature+, -, ? correlation
Definerelationship
Evaluationcriteria
Worked example
4. Rating of product characteristic to user need
use rating scale: 0, 1, 3, 9
none – weak – moderate - strong
Worked example
5. Weighting of product characteristic
Sum (weighting for each product characteristic) x (ranking of user need)
e.g., ‘centre of gravity’ = (1x10)+ (6x3) + (5x3) = 43
Worked example
6. Target values
Relate product characteristic to relevant Standards (where appropriate) and enter Standard into matrix
e.g. EN344 for safety shoes
Worked example
7. Technical and customer analysis
Evaluation against Standards;
Evaluation against competitor products;
Evaluation from user trials
Another Example
Technologies to support mobile learning Set of product characteristics
Defined by technologists within group From communications to displays
Set of user requirements Defined by learning theorists From learning activity to problems
RequirementsActivitiesEpisodesProjects
AdaptLearning objects
Learning OutcomesBreakdown
Mobility
Rank84376125
Conte
xt a
ware
Use
r model
Reca
ll conte
xt
Augm
ente
d re
ality
Adaptiv
e in
terfa
ce
Renderin
g x
device
Conte
nt m
gt.
Annota
tion
Cre
atio
n
Curricu
lum
mgt.
Text
Audio
Pictu
re
Modera
tion
Peer-to
-Peer
Bro
adca
st
3 8 8 3 3 3 1 8 8 3 3 8 1 8 3 81 8 3 1 1 1 3 3 3 1 1 3 1 8 3 81 3 1 1 1 1 8 1 1 3 1 1 1 3 3 30 3 0 1 1 1 8 1 1 8 8 8 1 1 8 88 0 1 3 3 3 3 1 8 8 3 8 3 3 8 81 1 3 1 1 1 8 0 0 8 1 3 1 8 3 81 3 8 1 1 1 3 0 0 1 1 8 1 8 8 83 8 1 3 3 3 1 8 8 8 8 8 3 3 3 8
97 173 109 74 74 74 137 132 174 191 148 242 58 169 183 273
Evaluation criteria
Standards and Criteria
Lecture Objectives
• Discuss the types of risk facing development projects
• Introduce techniques for identifying and managing risks
Mainly looking at IT Software projects, but risk management is relevant to all development projects
1
Defining Risk
• In product development, risk is:
The potential for an event (which would negatively impact the project) to occur
• Not the same as a problem
• Risk is probability / uncertainty:
a) of event happening
b) of impact of event
2
Is Risk Management Important?
1. Risks can lead to project failure
Failure is the “…perceived inability of the development project to meet the requirements or expectations of various
combinations of organizational stakeholders.”Ewusi-Mensah (2003)
Can judge success against:• Cost• Schedule• Technical Performance• Political factors• User acceptance
3
2. Risks can lead to project abandonment
“When the expectations of any of the stakeholder groups are unrealised, this can create a situation
where management are compelled to terminate the project prior to its installation and operation.”
(Ewusi-Mensah, 2003).
4
Is Risk Management Important?
A KMPG study in 2002, which covered 134 listed companies in the UK, US, Africa, Australia and Europe, reported that 56 % of firms had to write off at least one IT project in the
last year as a failure. The average loss incurred as a result of these failures was about €12.5m, with single biggest
write-off costing almost €210m.http://www.theregister.co.uk
5
IT Project Failure Rates
• UK National Air Traffic Control System (NATS)
• London Ambulance Service Computer Aided Despatch (LASCAD)
• Barclays Online Banking
6
Some Examples
In the press…
NHS’s £12.4bn National Programme for IT (NPfIT)
Channel Four ‘Dispatches’ (Monday 26th February)
Private Eye Special Issue (27th February)
NATS
“The UKs National Air Traffic Control system (NATS) finally went live in 2002, 15 years after it was conceived and six
years after the first promised date. The system took longer to plan and build than it will be in operation.”
http://www.computerweekly.co.uk
NATS has subsequently been plagued by:
• Equipment failure • Faulty software• Financial difficulty
http://news.bbc.co.uk
7
LASCAD
Computer Aided Despatch (CAD) system located ambulances and automatically communicated with them
It could not keep track of ambulance location or status, this led to error messages which slowed the system down
Public repeated calls as the delays increased
Entire system collapsed and operators reverted to manual system
Delays put public at risk
Chief Executive of LAS resigned8
Barclays Online Banking
Customers found that when they logged into the service they were able to read details of other people's bank accounts
Happened when two customers logged on to the service at exactly the same time
Barclays thought this was so improbable that it would never happen
Software was upgraded but resulted in loss of consumer confidence in online banking
9
3. Risks are not inevitabilities
We can do something to reduce the likelihood of risks occurring and the severity of their impacts if they do…
Risk Management
Is Risk Management Important?
10
Risk in the Development Process
Project will face different risks throughout the development process
E.g. Waterfall Model
11
Business Requirement
s
Design
Implementation
Testing
Installation
Maintenance
System Requirement
s
Verify
Transform
Transform
Transform
Transform
Transform
Transform
Verify
Verify
Verify
Verify
Verify
E.g. LASCAD
Project Abandonment
Requirements Stage Unrealistic project objectives Changing requirements Lack of executive commitment Lack of end-user commitment
12
Factors Critical to project abandonment (Ewusi-Mensah, 2003)
Project Abandonment [2]
Design Stage Unrealistic project objectives Changing requirements Inappropriate project-team composition Project management problems Inadequate technical know-how Problematic technology / infrastructure
E.g. NATS
13
Factors Critical to project abandonment (Ewusi-Mensah, 2003)
Project Abandonment [3]
Implementation Unrealistic project objectives Changing requirements Cost overruns and schedule delays Project management problems Inadequate technical know-how Problematic technology / infrastructure
14
E.g. NATS
Factors Critical to project abandonment (Ewusi-Mensah, 2003)
Risk Management
“The early identification and resolution of the project’s high-risk elements.”
Two main elements (Boehm, 1991) :
i. Risk Assessment Risk identification
Risk analysis
Risk prioritisation
ii. Risk Control Risk-management planning
Risk resolution
Risk monitoring
15
Risk Management7 Step Process (Fairley, 1994):
• Identify risk factors
• Assess risk probabilities and effects on the project
• Develop strategies to mitigate identified risks
• Monitor risk factors
• Invoke a contingency plan when a quantitative risk factor crosses a predetermined threshold
• Manage the crisis by possibly drastic corrective action if the contingency plan fails
• Recover from a crisis, e.g., by rewarding personnel, re-evaluating schedule and resources
24
Can identify risks through
Brainstorming
Checklists / taxonomies
Formal risk assessment methodologies HAZOP (Hazard and Operability Study) FTA (Fault Tree Analysis) FMECA(Failure Mode, Effects and Consequence Analysis)
Risk Identification
16
Boehm’s top 10 risk items:
• Personnel shortfall• Unrealistic schedules and budgets• Developing the wrong functions and properties• Developing the wrong user interface• Gold plating• Continuing stream of requirements changes• Shortfalls in externally finished components• “ “ “ performed tasks• Real-time performance shortfalls• Straining computer-science capabilities
Risk Identification [2]
17
Risk Analysis
Qualitative - Risk Matrix
18
Consequences
Likelihood
Tolerable Major Catastrophic
High Moderate High Extreme
Medium Moderate High Extreme
Low Low Moderate High
Depends on understanding risk and interpretation of level of consequence
Can be complicated, with multiple consequences for each risk
Risk Analysis [2]
Quantitative - Risk Exposure (Boehm, 1991):
RE = P(UO) * L(UO)
RE is the risk exposure, P(UO) is the probability of an unsatisfactory outcome and L(UO) is the (financial) loss to the parties affected if the outcome is unsatisfactory.
Can all risks be described in terms of financial loss?
19
Risk Exposure useful for risk prioritisation:
• Rank risks according to the estimated level of exposure;
• Compare courses of action - produce a decision tree;
• Prioritise testing.
Risk Prioritisation
20
But, is risk analysis an exact science?
Barclays Online Banking – Risk was thought too unlikely to require action
Risk Management Planning
Plans for handling risks, for example:
Risk Avoidance
Reduce the probability that risk will arise
E.g. Thorough requirements analysis
Risk Reduction
Reduce the impact of the risk on the project
E.g. Outsource development to 3rd Party
or
use COTS (commercial off the shelf)21
Risk Resolution
Implement risk management plan
Example techniques:
• Prototyping
• Simulations
• Benchmarks
22
Track project progress towards risk resolution and take action where appropriate
Techniques:
• Milestone Tracking
• Top 10 risk item tracking
• Risk reassessment
Risk Monitoring
23
Risk Jigsaw
25
Business Risk Possible Action
Understanding Education
Buy-in/Commitment Involvement
Changes Contingency Planning
1. Business Risks
26
2. Development Risks
Development Risk Possible Action
Estimating and Planning Project Management
Staff Turnover Staff Management
Development Tools Replacement
27
3. Architecture Risks
Architecture Risk Possible Action
Technical Competencies Outsource, recruit, train
Technology Platforms Match
Technology Life Cycles Postpone & Research
28
Risk Jigsaw
29
Other Risk Factors
Software Project Risk Factor Groups (Barki et al., 1993):
Newness of the technology Application size Lack of expertise Application complexity Organisational environment
Any more factors? New Markets or new customers Unrealistic budget constraints Competitors Cost / Benefit
30
Case Study
Customer Registration Software at a Mobile Telecoms Company
Background New company - about to launch and start registering new customers.
Existing CRM software could not register customers on the mobile network and company databases.
31
Background (Cont.)
At company launch, customer data captured on spreadsheets and loaded manually - created problems for:
– Order fulfilment;
– Billing customers;
– Customer Relationship Management;
Bad publicity for the company.
needed to quickly design and build a registration system that could be operated from multiple locations.
32
User Groups for new software:
• Sales (call centre, online, in-store)
• CRM (call centre)
• Billing
• Credit Analysis
• Fraud
• Provisioning
• Logistics
• Customers
33
Business Climate at Start of Project
No overall business owner;
No business project manager;
No formal documented Business Requirements;
Deadlines ‘set in stone’ by senior management;
Many User Groups not identified;
All Stakeholders not identified.
34
IT Development Climate at Start of Project
In-house development, despite lack of experience;
No formal development methodology;
No documentation of specific deliverables;
No formal Specifications documentation;
Little involvement of users and stakeholders during design, build and test;
High turnover of developers - all short-term contractors;
Tight budget constraints;
Other late high priority projects.
35
During Development
Business
No coherent business plan for product - frequent last minute ‘top priority’ requirements;
No monitoring of IT Development by business stakeholders;
Lack of buy-in from business owners – unaware / did not see it as a high priority.
IT Development
Little contact with business users / stakeholders;
No Unit Testing of new software.
36
Risk Jigsaw
37
Delivery
Business
Constant addition of new ‘critical’ business requirements;
Stakeholders: no idea of functionality being delivered, or when;
Users: Didn’t know how to use system, no procedures on how to perform business functions;
No structured User Acceptance Testing – often missed out;
No coherent business processes for system use.
38
Delivery (cont.)
IT Development
High turnover of software versions inadequate testing;
Code often resulted in catastrophic failure during final testing or installation;
Defects rejected as functionality “not part of original requirements”;
Problems fixing defects: “X wrote that bit of code and he’s left the company, no-one else knows how it works…”
39
Measures of success
Performance
Problems for all user groups
Customer Registration initially took 40 minutes
Problems for Fraud prevention
Compatibility problems with other systems
Provisioning delays
Schedule
‘Slippage’ of all deliverables
40
Measures of success (cont.)
Politics
Stakeholders often rejected project deliverables;
Reliability
System instability;
Cost/Benefit
Contact Centre system not extensively used;
Impact on Users
Inability to use software
Other Effects
41
Lecture Summary
Many ways to measure project success/failure;
‘Success’ decided by users and stakeholders
Several methods available for identification / management of risks to projects;
High failure rates of IT projects indicate that risk management is not easy or often well done;
Require support from all levels / areas of business for project to be a success.
Time spent on risk identification, analysis and management pays off
42
References Remenyi, D. (1999) Stop IT project failures through risk management, Butterworth-Heinemann. Boehm, B.W. (1991) Software Risk Management: Principle and Practices, IEEE Software,
Vol. 8, No. 1, pp. 32-41 Ewusi-Mensah, K. (2003) Software development failures, Massachusetts Institute of Technology. Fairley, R. (1994) Risk management for software projects, IEEE Software, Vol. 11, No. 3, pp. 57-67
43
