Embed Size (px)
Citation preview
Disclosure to Promote the Right To Information
Whereas the Parliament of India has set out to provide a practical regime of right to information for citizens to secure access to information under the control of public authorities, in order to promote transparency and accountability in the working of every public authority, and whereas the attached publication of the Bureau of Indian Standards is of particular interest to the public, particularly disadvantaged communities and those engaged in the pursuit of education and knowledge, the attached public safety standard is made available to promote the timely dissemination of this information in an accurate manner to the public.
इंटरनेट मानक
“!ान $ एक न' भारत का +नम-ण”Satyanarayan Gangaram Pitroda
“Invent a New India Using Knowledge”
“प0रा1 को छोड न' 5 तरफ”Jawaharlal Nehru
“Step Out From the Old to the New”
“जान1 का अ+धकार, जी1 का अ+धकार”Mazdoor Kisan Shakti Sangathan
“The Right to Information, The Right to Live”
“!ान एक ऐसा खजाना > जो कभी च0राया नहB जा सकता है”Bhartṛhari—Nītiśatakam
“Knowledge is such a treasure which cannot be stolen”
“Invent a New India Using Knowledge”
है”ह”ह
IS 11289 (1985): Code of practice for performancemonitoring of computer based systems [LITD 14: Software andSystem Engineering]
IS :11289= 1985
Indian Standard
CODE OF PRACTICE FORPERFORMANCE MONITORING OF
COMPUTER BASED SYSTEMS
Computer, Business Machines and Calculators SectionalCommittee, LTDC 24
Chairman RepresentingDR N. SESHAGIRI Department of Electronics, New Delhi
MembersSHRIR. P. AHUJA Computer Maintenance Corporation Ltd,
SHRI C. K. BAPIRAJUNew Delhi
State Bank of India, BombaySHRIS. K. BHATIA Bharat Heavy Electrical Ltd, New Delhi
SHRI P. V. C. CHELLAPATTTRAO\ -—.. -. ..___,
DR V1jAYp. B HATKAR Kerala State Electronics Development. Corporation Ltd, Trivandrum
DR S. N. S. RAjASEKRAm( Alternate )DR C. R. CHAKRAVARTHY Ministry of Defence ( R & D )SHRIK. N. DHeER Indian Airlines, New DelhiSHRIS. P. GOTHOSKAR Reserve Bank of India, BombaySHRIDEEPAKGUPTA Tata Electric Compantes, Bombay
SHRIR. M. NAIR(Alternate )DR MATHAIJOSEPH
DR S. RAMANI ( Alternate)DR A. LAHIR[
NCSDGT, Tata Institute for FundamentalResearch, Bombay
I)epa:t~:;t of Science & Technology, New
Life Insurance Corporation of India, BombayHindustan Computers Ltd, New Delhi
,)
SHRIN. -LAKSHMANANSHRIARJUNMALHOTRA
SHRIARUNMEHTA( Alternate,DR S. C. MEHTA Steel Authority of India, New DelhiSHRIS. L. N. MURTHY Bharat Electronics Ltd, Bangalore
SHRIK. S. PERINANAYAQAM( Alternate)SHRIS. K. PANDBY Internatta~~:mputers Indian Manufactures
SHRIH. DA$( Alternate ) P
SHRIV. K. R. PRABHU Development Commissioner ( Small ScaleIndustries ), New Delhi
SHFUM. RAMAKRISHNAN( Alternate )SHRI G. RAGHUKUMAR DelhiD~~th & General Mills Co Ltd, New
SHRI A.N . PAWAR ( Alternate)
( Continued on page 20 Copyright 1985
INDIAN STANDARDS INSTITUTIONThis publication is protected under the Zndiunt70pyright Act ( XIV of 1957 ) and m.production in whole or in part by anY means except with written permission of thepublisher shall be deemed to bean infringement of copyright under the said Act.
( Continuedfrom page I )
Members Re~esea~ing
DR V. K. RAVINDRAN PSI Data Systems fit Ltd, BangaloreSHRIV. \. ~CSHpA@B ( @~eiv@e )
lltoF R. SAD~NANDA Computer Society of ?ndia, BombaySHRIAsms SEN ?tidiiin 9t@isth@’Iqs@u@, C@uttaSHRI UMESH P. SHAH 6RG Sy$tefis, Vadodara
%-nu P. K. SRIDHARAN( Altewmta ) .,,
SHRIM. SFIANKRALXNGAM Director@e Geneg?l of Suppli s & DisposalsA
SHRIK. L. GARG ( Alternate )( Irispection Wihg ), New elhi
SHRI C. G. SUEtRAMANYAN Electronics Trade and Technology DevelopmentCorporation Ltd, Ncw Delhi
SHRIISHWARDUTr ( Alternate)SHRIT. N. SWAMY Elect ronics Corporation of India Ltd, ~y~erab@$WH V. R. U~NIRA~AN Telecommunication Research Cwitre, New D61hiSHRIK. M. VISWANATHAN Hindustan Teleprinter Ltd, Madrqs
SHRIS. DBVARWAN( Alternate ~S HRIN. SRINPMSAN, Director General, 1S1 ( Ex-o#icfo Member )
Director ( Electronics)
Secretary
SHRZA. & RAWATDeputy Director ( Electronics ), 1S1
Information Systems and Procedures Subcomtnittee, LTDC 24:3
SHRI C. K. BAPIRAJUSHRI S. K. BHATIA
SHRIH. P. SAXEt+ASHRIS. P. GOTHOSKARSHBI ~. M. N. KAPURDR R. SHAINKARSHRIR. THJACjARAJAN
DR N. VXJAYADITYA
( Alternate)
State Bank of India, BombayBharat Heavy Electric.ai Ltd, New Delhi
Reserve Bank of India, BombayCentral Statistical OrganizatiW, New DelhiIndian Institute of TechWlagy, KanpurDepartthlt of f%iinsce. and TUGIMM1OSY,’New
National Information Ccntrc, Flew Delhi
‘2
4!!
, $’,,
).
t
/“
,. -
m: Uwx 1*
Indian Wmdkmd
CODE OF p~~IQQ ~~gpG~PQJQIi$NCII b@N1~QgJNG W
COMPUTER BAflED SW!#rf$Mfj
0. FOR EWO-RD.- r..,.- *.
0.1 This Indian Standard was adopted by the Indian sp~rd?Institution on 29 April 1985, after the draft finalized by the computer,~Usingxs Machings and Calculators S@ional C mittge had been
Yapproved by t~e Electronics and l’elgcotn~~qicatiqn jviaion QNU@.
0.2 This code of practice applies to all computer-based systems andcovers following aspects:
a)
b)
c)
Gives guidance in carrying out a detailed examination of thesystem that is to be monitored. It ic@ti$im lmy ~tiwcts gf #h@system, its usage and other sy?t?p$ @N ~~.We:$~? ~tlw o,l?!X%lt-ing environment. Guidelines and check hsts are provided foreach of these qspects to assi~t ,the yser i~ idgn~ifyj~g @gm.w@s @change that can affect performance;
lslentifi~s a yarig%y Qf monitoring techniques that may be usedto measure t&se changes in performance; andGives guidance on the ~ocym&M@n of p&forrn4ace moai~in~information.
0.3 This code gives guidance to enable organizations to:
a) assure the continuing quality of a computer-based system in
d) detect faults so that action can be taken to qvo@ faulj pro-pagation.
4.4 This code assumes that the system:
a) was adequately tested and documented in accordance with?@?van! $twdards,
b)c)
performed as required under ns~al. qmratkg conditions, andwas initially schedulable within the COQSW&@.W by
other systems and manufa.ctur#& design.
3,,.! “
I
,.
..
,?. ‘,:
IS :11289-1985
0.4.1 This code also assumes that, in service:
a) the system runs under effective configuration management, and
%) documentation is ‘maintained.
0.5 This, standard is based on BS: 6238:1982 ‘Code of practice forperformance monitoring of computer based systems’, issued by the BritishStandards Institution.
1. SCOPE
1.1 This code of practice provides guidance for carrying out exami-nation of the system with a view to monitoring and documenting itsperformance.
2. TERMINOLOGY
2.0 For the purposes of this standard the terms and definitions as givenin IS: 1885 ( Part 52) Series* shall apply.
3. KEY ASPECTS OF THE SYSTEM BEING MONITORED
3.0 The system monitoring requires a detailed examination as under forall likely causes of a change in performance and the ways in whichchanges to-other systems sharing the same resources that may affect theperformance of the system being monitored.
3.1 Applications Software
3.1.0 General — Some elements of change may have been anticipatedduring the design of the system but serious effects can arise fromchanges that occur, which were not visualized ‘in the original specifica-tion.
3.1.1 Elements of Change — Among the elements of change thatshould be examined, the user should note that:
a) standards of accuracy can affect processing times,
b) security controls can extend input and output duration,
c) validation checks can extend input and output duration,
d) changes in coding of data can affect input and output times andstorage capacity required,
●Electrotechnical vocabulary: Part 62 Data processing.
4.
,,.’
.’ i,P
,1
e)
f)
g)h)
j)k)
m)
n)
IS: 11289-1985
changes in code structures can affect input and output times andstorage capacity required,
changes in the structure of the -data base and files can affectinput and output times and storage capacity required,
changes in input method/documents can increase input times,
changes in output method/documents can increase output times,
changes in parameter values can change processing times,
changes to algorithms or formulae could change processingtimes,
some changes may not be reflected in the procedures, and
some changes may not be reflected in the documentation.
3.2 Operating and Support Software
3.2.0 General – Changes to the operating and support software andsystem can have a considerable influence on the efficiency and beha-viour of a system. Such changes are quite usual for various reasons(for example, cor~ection of faults or the improvement of facilities ) buta careful examination of their implications should be made to ensurethat no harmful side effects will occur.
3.2.1 Elements of Change
3.2.M Operating software — Operating software includes all thesystem, facilities used in the course of running application programmed.The xnain consideration is that an alteration to a system facility mayaffect some or all application programmed, or ruqlules, and it may notbe practicable or desirable ta prom-reference facilitks with applications.Hence changes canriot nece~s~ri}y bv iguaratnttmd to have no side effectsand each one should be treated on its merits..
The main system facilit~eq that come qnder this consideration art%
a) system library, holdl~ng commonly used modules;
b) fde handling system ( changes here almost certainly affect allapplication programme$ );
c) job control language ( pew or added commands or formats);
d) input/output; and
e) scheduling. ( Changes to the priority strategy will affect run-times, elapsed times dr even what time of day or night anapplication will be run, These in turn will affect the run-timecosts. )
6
“m”’sIim ~ Wfi
24.M2 I&lppOrt software!
a)
b)
General — This software includes all the $yifehi ‘fadi.litiei used inthe cdurse of p?ogtaidti~ pJ!@ariM&i; Thd imde!MtMe! tetilt ofchahges iti this are’a is the necessity fo# #ib&WW6F retrainingand the possible reduction in s@tware qsali%y and &Wo@putuntil the changes are fully understood.
Language translator ( compilers) — Special consideration shouldbe given to this f~eility. Chariges in ~his arda mdy have a verysignifkatit ehed cm application SYS*MSperfofrn$itwe and shouldbe carefully vetted as they may not always be appareiit. Generally,changes will be manifested as version updates or the samelanguage from a different supplier ( probably giving faster run-time code, or better sto~e utili~titioti, of jtmt im@ov@ f$cilhies ).Even if a system has -been extensively acceptance tested, thereremains a need to understand that ii M&ently iti~ussible tbobtain 100 percent assurance that errors have been eliminated orall irnp~icatioris fully appreciated. Sod& of the rnbre obViousfeatures that may cause dit3ibuIty oh ah otlh5fwlse idetttical tfM-Iator are syiitax, sernantib% rt%seiv~d Words $uid keyw~rd$,tratislhtor dirdctivek atid linkid&
Some of the less obvidtis fe~&di thttt shdtild W Mmsick%retlare as follows:
1) Arithmetic — Although these probleifis shoiild be’ rescilVedduring aceeptanoe testimig of the new trandatck, explicitchecks should be made on the fb~lowing p@i~ts to ensureconformity with previou$ seft$vare
[i] Turri6ation-and .rohridiritj,
ii) Th& result and remaint.let fmm the division algorithm,
iii) Equality tests foi flcdttifig 6? iiiixc+ddMrnWt$ ~hd
ivJ Overfi&v Ihhifi slibtild M be M tlitlfi pi’eticttii Mbits.
2) Ex@ks&btt$ - fii$ featuf~ k d Whihk #MBkdl M (fi) and
3)
the following points should be checiced:
i) Order of evaluation should B(5 iddht~dd,iij’ Miilid ‘i~pd’ d6iiVerUbii i%fidti~’@#@it16tM@l fiiMd@ and
iii) Intermediate state limits should net W Wss’than pf6viouslimits.
I%@r/hres mid jliwk+ionw - ‘WO WW#birJW $hould be&hduiWWi) Order’ of parameter evaluation should be & #ifi.?E;and
‘+‘6 , ,“. .,
,,
f+
.,,.*.
,
I
ii) The parameter mechanism should behave in the same, @y\for the Mtne cases ( by t~fettmce, valti$, et~,). l’~fi be a:
viour is particularly impcMarit for def’titdt eastx.
4) Optimization — A different dptitniktttitm cim give differentresults for some application ~rogrammes.
5) Error handling — Changes caa cause some temporarymisinterpretation.
c) Other support software — This category includes facilities suchas:1) time sharing command structure,
2) debugging taols,
3) editor, and
4) linker.
Changes to the linker can also interact with the system library, wbicihis considered under operating software ( see 3.2.1,1).
3.2.1.3 Software reconjlkuration — Altltcmgh there may be tidchanges to the products that form the operatitig and supporting soft-ware, they may be reconfigured in such a way as to affact the perfor-mance of a system. Some of the aspects that need to be considerwl inthese circumstances are:
~) mapping of software on to cliffere@ types and sps~s “of rqedj#devices,
b) m~nimiz~~g the ~ssiblq pvsi~ea~p Qf ~@i~gj
c) change in programme loading format and method,
d) m&@Wibikof t@tt$& ~~tM@W$
e) selectioii of iiew rutihiii$ rnddes stich as clibice of 10$S qrormessages or full listing~i
f) change in normal resid~~e and running ver&RM# of 80fkwM~,
g) change in ww of fi%@@$ -w w.%*h) charige itt tiik!.df @Wttt~Y!fe~f~~ @, ~~$elt$, ~d
j) change in use of applicat~op programme diagnostics.
3.3 Hardware Configuration
I 3.3.0 General — The performance of a com@ter-based s stern, cgn be1, limited by me w MU h@MWm r@W#&e, iA rkti$l~ W t e dkif3$ura-
tion tan rwcwtk * .if#iW@i# &n$ iHk14 MdMu@ (M.M&RMiw$ W#
?
IS :11289-1985
limitation for another. The effect of configuration changes should beconsidered under the following heading:
a) Processor ( CPU ) capacity,
b) Storage devices, -and
c) Input/output devices.
3.3.1 Elements of Change
3.3.1.1 Processor ( CPU) capacity — A change in the speed orcapacity of the processor unit can put pressure on the ability of storageinput or output or any combination of these devices to cope with theresultant changes in volume of data or speed of data flow.
3.3.1.2 Storage devices — A balance should be maintained betweenthe amount of main storage and the number of peripherals attached.Increasing the amount of main storage can allow more programmed to runccmcurrently, but will increase the loading on existing peripherals anddegrade the performance of existing systems accordingly. Conversely,increasing auxiliary storage can also reduce the performance of acomputer-based system.
3.3.1.3 Input/output devices — In thet context of this code, input/out-put devices include automatic control devices, VDUS and sensors, etc.
Increasing the speed, number or function of these devices can affectthe performance of a computer-based system. In particular, increasingthe number or changing the usage mode of VDUS can increase thecommunications overheads with a consequent impact on other systemsareas.
Greater peripheral capacity can allow more programmed to run con-current y, increasing CPU load, channel load and operating systemoverhead.
3.4 Hardware Ageing
3.4.0 General — Ageing can be a serious problem with those compo-nents of a computer configuration w-here motion and hence wear canoccur. The two main areas that should be considered are:
, a) electromechanical devices, and
b) magnetic media.●
3.4.1 Elements of Change
3.4.1.1 Electromechanical devices — These devices become lessreliable with age, decreasing the mean time between failures, and hence
8
..”,
IS :11289-1985
increasing the elapsed time for jobs. Despite preventive maintenance,equipment reliability can eventually drop below the acceptance leveldemanded by the response-time of the system,
3.4.1.2 Magnetic’media — Tapes and ‘disks are subject to deteriorationin use. This deterioration manifests itself in an increase .in the numberof read/write errors that occur. The number of hidden re-tries by theoperating system can greatly increase the run-time of the programme.Excessive use of a short length of a full tape reel increases wear andshould be avoided.
3.5 System Inputs
3.5.0 Gerteral — All computer systems should check the validity andintegrity of the input data. However, there are many changes that canbe made to the input data that may not be detected, and such changed
~ data may have a wide range of adverse effects on the performance of theI system.
a)
b)
c)
d)
e)
f)
g)h)
3.5.1
The input data should be considered under the following headings:
Manual data collection systems,
Data preparation procedures,
On-line data entry,
Data validation,
Parameter data,
Data brought forward,
Real time data, and
Transmission channels.
Elements of Change
3.5.1.1 Manual data Collection _ Most commercial batch processingsystems rely heavily on some form of manual data collection. The dabcollection can range from filling in forms or marks — sensing cards t~collecting magnetic encoded strips or pre-punchd tags.
\
The system performance can be affected by the following factoramongst other things on this stage:
a) New or temporary staff who have not been adequately trained,
b) Instruction manuals not kept up-to-date,
c) A new or modified form design not properly understood by oneor more of the staff,
9
.,
.
,.
,,,
,,,
1S :112$9-1985
d) A new or modified means of expressing the data not properlyunderstood by one or more of the staff, and
e) A cut-off point for end of month or end of year not clearlydefined.
The performance of a system can also be affected by fluctuations inthe volume of-data. If there are not sufficient data collection staff todeal with peak loads a back-log builds up. When this is cleared byovertime or additional staff it can have disastrous effects on systemperformance.
3.5.1.2 Changes in data preparation procedures — Changes in theseprocedures are likely to have a significant effect on system performance.A cut in verification rate can result in more errors which means moremachine time on error printing plus ( usually ) a second input whencorrected. Also, inadequate training of new staff can cause problems if,for example, a punch operator is not aware of the installation standardfor writing the letter ‘O’and the figure zero. Changes in input mediafrom, say, punched cards to disk can cause problems if not handledcarefully.
3.5.1.3 On-line data entry — There needs to be adequate controlover personnel permitted access to the terminal. It is particularly impor-Iant that new personnel should be properly trained in the correct way toenter data, and existing personnel are retrained when equipment ischanged. It is essential that people who have the facility to interactwith an on-line system are sufficiently familiar with the system and whatit has been designed to do so that the implications of human interventionare thoroughly understood.
3.5.1.4 Data validation — Inadequate checking and cross checkingof data being input to a system can result in all manner of problemsthat may affect the performance of the system in unpredictable ways.
Very few systems can afford to carry out total validation on allinput data. All systems should, however, check that data are valid withinreasonable limits. What is reasonable can change, however, as a resultof modifications to the system and erroneous data can remain undetectedby the existing validation routines.
- 3.5.1.5 Parameter data — Parameter data can be used for a varietyof purposes including:
determining whether a programme functions in a weekly or amonthly mode,
establishing data validation limits,
setting constants or literals, and
10
. .4
IS :11289-1985
d) determining the extent and nature of processing to be undertakenby general utility programmed.
The very nature of parameter data makes it difficult to validate oninput because data that are within allowable limits can be incorrect forthe intended purpose.
Where parameters are used to determine. the extent and nature ofprocessing by generalized utilities ( for example, report generators ), the
I run time may vary greatly depending on the parameters used. Misuseof such facilities can result in severe impact on other system areas andtherefore adequate parameter valid ation and control over these runs isrequired. It is recommended that arrangements should be made forthe data to be independently checked from a different source wheneverpossible.
Changes in staff, inadequate training of staff responsible for enteringparameters acd out-of-date instruction manuals are likely causes of
error.
3.5.1.6 Data brought forward — Data that are brought forward maycome from a previous run of fhe system that is being monitored or fromanother system. It is necessary to ensure that the system being moni-tored checks that the correct data have been brought forward. If thedata brought forward have. come from another system, checks shouldbe made on the validity of the data by the system being monitored.Likely causes of performance degradation include:
a) amendments to the programmed or the system that wrote the ~data,
b) operational probiems such as malfunction of the device on whichthe data were written, and
c) an additional unscheduled run of the system that wrote the data.
3.5.1.7 Real time data — Real time data are data that are intendedto represent the immediate measured state of the environment, forexample, data from sensors, analogue to digital converters, instrumen-tation and control equipment, radar systems.
In many cases the data source can itself be a complex relatedsystem with its own error and integrity checks so that main concern ofthe system being monitored will be the data transmission errors andpossible periodic message integrity checks ( for example, byte counts ).
When the control of the data source comes within the province ofthe system being monitored, then in addition to the data transmissionand message integrity checks, the system should include periodicintegrity/calibration checks or runs on the data source.
11
I
/
,’:
4
@ :11289-1985
It should be emphasized that, however, consistent the data appear tobe to any integrity tests, if the data source is inconsistent with theenvironment the information content of the data will be impaired and thesystem performance will .be degraded accordingly.
3.5.1.8 Transmission channels — Although operation of on-linesystems should be transparent to users, various levels of error checkingand retransmission will be occurring within each system, An increasein the number of retransmissions will occur if the transmissiim pathbecomes ‘noisy’ or unreliable, giving rise to increasing delays in the datatransmission. This aspect should be monitored and warnings generatedwhen the degradation reaches a predefine limit.
It should also be noted that changing the physical nature of a trans-mission channel -can affect not only the bandwidth but also the type ofnoise or interference on the channel ( for examplej public/private, wires/ratio links, wire/optical fibre ) and the error checking/moriitoringroutines may have to reflect this in order to minimize system degradation.
3.6 System Usage
3.6.0 General — Changes in th~ usage of the system being monitoredcan be reflected in modifications to the significant characteristics ofthe system inputs, processing or outputs or any combination of these. Insome instances, the impact of a change will be readily apparent, forexample, doubling the number of employees paid by the system willincrease the size of the payl 011run. In other instances the effect cannotbe so easily predicted, for example, a change in an organization’s salesdiscount structure may result in customers changing their orderinghabits, thus affecting the sales invoicing routines.
Prominent among the aspects of system operation likely to beaffected by changes in usage are:
a) increases or decreases in volumes of data,
q) revision of timing,
c) revision of quality controls, and
d) misuse of the system.
3.6.1 Elements of Change
3.6.1.1 Increases or decreases in volumes of data — Increases involumes of data can be expected to pose a potential threat to systemperformance while decreases will normally be expected to have either abeneficial effect or none at all. Increases in volume can occur in, forinstance, numbers of documents, numbers of items in a document,
12
,.
,,
.
,,.
-,: !.< >,
IS : 112S9 - 198!$
numbers of items on a file, numbers of characters per data item, numbersand ranges of paranieters.
It is advisable to trace the possible effect of any change through thesequence of input, processing and output to ensure that all possibleeffects on the performance of the system are fully explored.
I 3.6.1.2 Revision of timing — Revisions of timing can include changesto deadlines for inputs or outputs and frequencies of computer runs.These changes can threaten maintenance of schedules by exerting pres-sure on any or all of the inputs, processing, storage or output facilitiesavailable at the time.
3.6.1.3 Revision of quality controls — System performance can beadversely affected by revisions to the nature of the controls applied,including validation techniques, security dumps, access controls, boundarylimits, reconfiguration rules. Again, it is advisable to trace the impactof change both forward and backward through the input, processingand output sequences to determine what effect they may have on theoverall performance of the system.
3.6.1.4 Misuse OJthe system — A system can be run in a way notintended by the designer in order to obtain urgently required -infor-mation. Although such actions can appear to have been achieved quitesuccessfully, they can have serious and unforeseen secondry {effects on,.for example, data integrity, run frequencies, dependent systems and
1security.
# 3.7 Effects d Other Systems
3.7.1 General — For the purposes of this code, other systems areconsidered under two main categories:
a) Unrelated systems, and
b) Related systems.
An unrelated system is one that has no direct interface to thesystem being monitored but that shares resources with it. A change inuse of the shared resources can therefore affect the performance of anysystem that uses them.
A related system is one that has a direct interface with the systembeing monitored. A change in an interface or in use of the relatedsystem can therefore affected the performance of the system being:monitored. A related system can also include the characteristics of an.unrelated system, that “is, share resources.
13
,!, ,
t
~I
..._
IS :11289-1985
3.7.2 Elements of Change
system
a)
b)
c)
d)
3.7.2.1 Unrelated systems — The major items that should be consi-dered in these systems are those resources that can be shared with the
being monitored, as follows:
Processor — Any variation in the programmed that concurrentlyuse the processor capacity can have an effect on performance, inparticular the elapsed time and through-put. The variation maybe the introduction of a new programme, which increases theoverall load, or a change in priorities and scheduling, whichresults in a different mix of jobs. Increased usage of an existingprogramme or new arrangements for the use of the processor toact as standby or backup can also affect the overall loading andperformance of the processor.
Communications systems — An increased load on communicationlines and controllers by one system can affect the performanceof another system that shares the communication facilities. Achange in the configuration of the communications aspects of asystem should be considered as a change in the hardware of thatsystem and be reviewed under that heading. However, a commu-nications configuration change, for example, new connectionsor new line speeds, can affect the performance of the mainframeprocessing system and therefore affect the performance of othersystems, even if they have no communications capability.
File store and peripherals — A change in the size of usage offilestore can affect the performance of all filestore~traffic and havea significant effect on any associated system.
Re-allocation offilestore to different peripheral types can improve or degrade theperformance of the system being monitored even though it doesnot use the changed filestore. Reconfiguration of peripherals orcontrollers that are outside the systenl being monitored cancause changes in the priorities or se~ice requests within anoperating system and hence change overall performance.
Storage media — The most significant changes arise when themedia that form part of the system are directly changed. How-ever> a change in the size or organization of a media librarv. . .or handling mechanism can cau~e a change in the servic~provided. An increase in the media volumes, for example, packs,cartridges or reals, may also require a change in housekeepingand archiving practices which could affect the users of thevolumes.
14
IS: 11289-1985
e) Data preparation — A change in data preparation equipment,the number of staff employed or the workload can affect anyuser of the service and therefore can affect the total behaviourof all systems that rely on that service.
f) Operations — A change in the total hardware configuration orthe total workload or the number of operations staff can affectany system that is the responsibility of one group of operationsstaff.
3.7.2.2 Related sysrems — In addition to the major items listed forunrelated systems, the items to be considered in these system are thoseareas where there may be direct interfaces with the system beingmonitored.
a) Interface data formats — Any change in the format of data,even if not used by the system being monitored, can affect ~er-
b)
c)
d)
formance, for ex~mple, ‘i~creased ~ecord len@”h or new re;ordtypes. Data to be considered can be transferred in processormain store, through filestore, on magnetic media, over communi-cations lines or on paper. Change can be planned OPaccidental;
,Data sets — A change in frequency of operation can introducemore data volumes to hold the same amount of data and therecan be an indirect effect on performance;
Procedures — A change in operational procedures for one system-can change the procedures for another; and
New or changed system — A new use of existing data caninfluence the old system to which it is related, for example, datarejection causing a-requirement for more stringent vettiig of theoriginal input data.
4. MONITORING TECHNIQUES
4.0 General
4.0.1 This section identifies various types of monitoring techniquesand provides guidance on the purposes for which each is best suited.Information is provided on each technique and what it achieves. Generalinformation and guidance is also included on the implications of the useof each technique and of the types of resources which may be required.
4.0.2 Each system to be monitored and its environment are uniqueand will require a different combination of monitoring techniques inorder -to be cost effective. In the event that a change to a critical qualityis revealed by the chosen monitor, the user will need to investigatefurther, possibly with selective use of other monitoring techniques, in
15
...
/
., %. !e
,’
IS :11289-1985
Order to discover which of the possible areas of change has actuallycaused the deterioration in performance.
4.0.3 The prospective user should review his system and the totaloperational environment before introducing any monitoring techniquesand some of the questions that should be considered are as follows:
a)
b)
c)
d)
e)
Which are the critical qualities which require monitoring ?
What is the cost ( man and machine resources) of obtainingthe data ?
What is the ~ffect on the performance of the system of intro-ducing a monitoring technique ?
What is the output from the monitoring technique ?
What training is required in order to introduce and imdementa monitoring-technique and process its output ?
.
4.1 Use of Routine Records
4.1.1 Records should be kept as a routine management control of theday to day running of a computer installation. The information contain-ed in these records will very often provide the first indication of a chargein the performance of the hardware, software and human componentsof a computer-based system and, thence, of the system as a whole.
4.1.2
a)
b)
c)
d)
e)
f)
g)h)
j)
k)
.m)
n)
P)
Classes of information that maybe found useful inclu~e:
volumes of data handled;
system response times;
job run frequencies
achievements against run schedules;
equipment availability, utilization and idle time;
backlogs of work;
environment (temperature, humidity, etc);
errors in inputs, processing and outputs;
nature, frequency and duration of hardware faults;
nature and effect of software faults;
frequency and duration of power supply failures;
nature, frequency and duration of communication faults andtheir effects; and
change-control records.
16
j—--”—-””-----”-—.. ,,,,. .
,, *,,
..
4.1.3 Routine reports generatedchanges in performance to be detectltion of specific monitoring technilchanges.
4.2 Hardware Monitors
4.2.0 General — A hardware moniformance measurement capability a~computer systems. In simple terms :device that is physically connected 1and record its activity. It is calledmonitors the hardware, hut becauseto an item of software.
Whatever measurements might’ing a performance analysis study, tto determine the causes of good ormeasured utilization should be inv
4.2.1 Capabilities — A hardwarea) time or count the activity a
peripherals) in a computer
b) distinguish between; classtiexam hs, , the component I
ftrans tip, ,$*~tick dMGk $*
jc) me.&@ ~fhki#&@#$y ‘
p~~ti’atim’mounaccess device;
d) analy$e and’ display the:~using in-built minicompu%~
e) impose no ‘interference or !
4.2.2 When to Use a Hardwarebe usefully employed:
a) when the system hardware
b)
c)
d)
suitable for a software mo
when a software monitor ~not available,
when th-e overheads impacceptable, and
when measurementsare required.
beyon
/“
..!
,.,
.-
,.
‘4
IS :11289-1985
4.1.3 Routine reports generated from these records should enablechanges in performance to be detected. This may then lead to the applica-tion of specific monitoring techniques to identify the causes of thesechanges.
4.2 Hardware Monitors
4.2.0 General — A hardware monitor provides the most powerful per-formance measurement capability availabh? to assist in the monitoring ofcomputer systems. In simple terms a hardware monitor is an electronicdevice that is physically connected to the computer and that measuresand record its activity. It is called a hardware monitor, not because itmonitors the hardware, but because it is a piece of hardware as opposedto an item of software.
Whatever measurements might~be taken by a hardware monitor dur-ing a performance analysis study, they are rarely sufficient in themselvesto determine the Wuses of good or bad performance. The results of themeasured utilization should be investigated further.
4.2.1
.a)
b)
c)
d)
e)
Capabilities — A hardware monitor.is normally able tottime or count the activity of every device ( CPU, channels andperipherals) in a computer configuration;
distinguish between classes of activity within a device, forexample, the component parts of a disk access ( seek, latency,transfer, feed-back check );
measure the frequency of obeying identified instructions in aprogramme or the amount of time accessing each track of a directaccess device;
analyse and display the results of measurements in real time,using in-built minicomputers and VDUS; and
impose no interference or degradation to the system hardware.
4.2.2 When to Use a Hardware Monitor — A hardware monitor canbe usefully employed:
a) when the system hardware or software does not provide facilities
b)
c)
d
suitable fo; a software monitor,
when a software monitor compatible with the system hardware isnot available,
when the overheads imposed by a software monitor are notacceptable, and
when measurements -beyond the abilities of a software monitorare required.
17
4“
Eli : 1X2S9-MM ,
4,3 $oftwam Mo&ters
4.3.0 Gem?tal - ~ soffwake moliitdr is a db ‘ “,utdr ro#amrlW tkt?- Yresidek ifi a computer $ystt!fn, the task of which M o CO1ect ~flformatiun
about the way the system is being used over a period of time. There arethree categories of software monitor: programnw mmaitom, systemmonitors and operating system data logging.
All three categories use system resourcm, the extent of which sbaklbe ccmsidered when iht~rpreting results. it “isalso desirable to irwmpoxaterun time switches to pefmit them to be initiated, suppressed d U@selectively.
4.3.1 Programtie Mbnitors
4.3.1.0 General — Programme monitors re ort on the CPU u.tiliza-1tio~ of ag individual .progratnme. If it is Mteit ed to extract $i ~tttle more
capacity from a configuration then it ttxay well be worth considering itperformance twtduatiofi exercise concetifdting on the fiv6! to ten rtlosttimeconsuming Trogrammes. Progradirte tnonitors eah be obtainedcommercially for a relatively low sum, hut alternatively they can often beprovided by the individual programmer/insf.#llation. There are three typesof programme rtlotiitoi.
4.3.1.1 Prograrnme instruction – This instruction w count thenumber af tidies a rutitim? is ehtet’ed &tx’itig ~ ~rogiam”rne,
4.3.1.2 A ‘harnessy monitor — This mt$rlitoi codiptises a prdgrammecompiled into the subject programxne providing information in such formsas histograms of the frequchcy with whi& any instruction is obeyedor traces of branching paths.
443,1.3 Measurement by timing each in ftt%ction or ratttine — Thismeasurement can be done using either of the fblbvitig tnethods:
a) Ifitwwrgatian of thv &m tral procwswt /iW# ftiit~ @ use~ pro-grarnme — This ean impose severe overheads on the run~ing timeof tlie Progfmtnm$.
b) Activity sampling — This involves the random cwphwe of thecurrent contents of the ins~r,ucti~n Wuntef and. am either bedone with proprietary software or by using the centr~ processortiihff~ iirttifiipt f~~ilit~ to ac$tivti~d tlid i?Rt&iWfat M&ila* inter-vals. .Currently available software has the adWM&@ M in-builttmdmiques,to ensur~ th~ randcnw aa$tw of the amnph;
4.3.2 System Monitors
4.9.2.0 &h#fa[ + SjWMri miMit* W&t ifilttH.fMtiQd @i tfi~ per-formance of the system as a whole including chitttit~ &fi&@!ripheral
ill
--
,~,‘r ,
‘,,
M { 11289- l.%1$
tiiiliiatidii. They are essentially paeka@s wcwlfipg cm a saqdiag basissiinilaf to tli& pr6@~mme monitors deseribed “in4&l but cx~twi~ c?pieself tabl~~ and listfi maintained by the operitiag systerq skwing the statusof the sygtein at ~fi~ one time. Howeverj th6y deniand ~esources f’tom thesystem they are monitoring; amounts vary but.a.re likely io be significant.The copies produced are stored and subsequently analyied by a secondpackagd which prdduces a cumprehehsive series of reports dethili@g theutilization lt?vel-$ c$f the vtirious hardwafe com@wiksJ inckkdihg storeutilization add cMfral processor usage, brokdri cWtwn bdtweeii o@rttti@gsystenis mid user programrne. It is usual to obtain systikn software m@i-tors from ind&pefident software houses by purchase ok l~ase.
4.3.2.1 Capabilities — A system tioriit~t is
a) simple to use; and
b) capable of producing output records relating to the logicalcomponents of the system ( for example, usage 01’system filesis reported under file name rather than the physioal deviceconcerned ).
4.3,3 Operating System Data Logging
4.3.3.0 Generui — This facility can be ebhsidWtRl as a softwarernotiitor and the value of the infortiatidti pftwiddd tb the perfurmarwemeasureriitmt prdct%s cad be t%ormotis.
4.3.3.1 Capabilities — The data can:
a)
b)
c1
show the hardware resour’@b requ-ired to process a givenWork-1oad; ~
provide infornkltion for the diagtiosis df cfttt$e~@fvafkitiorrs ihsystem performance, for example, excessively high demand forparticular system resources during certain phases of the work-load will explain degraded processing timek; and
provide inforiiiatidii M d basis fdt lm$dieting k$vels uf berfor-Widce.
4i4 Ill SWWi&i! T@stih&
4.4.0 General — During acceptance testing the user should have ensured-that the system will meet his requirements. He t#~il iiiSt3 nt%clto repeat cettitin tests to ehsure that thq systeq continues to meet his.perl’hmonce rsqui~ments. ‘I&is task dan be divided intcit
ii~ c?hksiiridjjMilt &i? iiW&Wfe Coiitititti% W ftiht?i@t 66ft@dtljj, iittiii
L$)easuring that tM *s- as a wkie &m&es t~ proviike i4@-factory service.
19
IS :11289-1985
Both categories usually involve running bench mark programmed wherethe expected result has been predetermined, It is not necessary to carryout regular tests on the software as it does not degrade with use ortime. Any changes to the software should be subject to the normalacceptance test procedures.
4.4.1 The System Hardware — It is important to remember thatsystem hardware in this context includes power lines, voltage stabilizers,no-break systems and similar equipment. The bench marks for in-servicetesting of all the major hardware elements are usually provided by themanufacturer, who should also provide diagnostic facilities. The checksneed to be performed periodically for preventive maintenance purposes.In addition, certain elements of the hardware may need to be monitoredcontinuously where they are critical to performance.
4.4.2 The Total System — Testing of the performance of the completesystem involves constructing some form of bench mark test where the‘bench mark bears some known and close relationship to the usual workprofile. The user will need to consider which method to use to apply thebench mark, for example, externally through a peripheral device or inter-nally using a data base. Also, it is necessary to consider the method tobe used to measure the relevant characteristics. It should also be appre-ciated that, although greater accuracy may be achieved when the benchmark has a close relationship to the actual environment, the cost factorcan be significant and a more loosely coupled relationship may give amore cost effective indication.
“4.4.2.1 As the characteristics of the system will vary with the workload it is important to establish the effect of high load conditions onthrough-put, response time, and availability at different levels of load.It will also show how different failure modes will occur.
4.4.2.2 Suitable bench marks can be used to show that correctiveaction taken to fix errors has been effective. In a similar way benchmarks can be used to optimize a performance of a system by trading offone characteristic against another. This should be done by studying theeffects of various parameters under constant bench mark test and thenseeing the result of making discrete changes, that is, introducing highspeed memory or changing queuing arrangements to remove bottlenecks.
.4.5 System Audit
4.5.0 General — One of the objectives of a system and it is to checkfor changes in the performance characteristics of the system. Thesechanges can indicate a degradation of the system or alternatively may-berequired to show an improvement following some earlier correctiveaction. To audit a computer-based system it is necessary to perform
- i-. -
“.
20
j.,
IS :11289- 198!%
some checks both manually and by automatic means and then to com-pare the results with historical records. In an ideal situation, special.bench mark programmed can be run whose results would give an indicationof performance changes since they were last executed, these bench markprogrammed having been produced as part of the original system andacceptance tests.
In order to be -able to define precisely the reasons for a particularchange in parameter, the audits need to be comprehensive.
4.5.1 Utilization Time by Users — ‘This information wili show whichareas of the organization are increasing their use of the system and thosethat are in decline. In-a teleprocessing system this information can sug-gest that a redistribution to access nodes may be necessary.
4.5.2 Mean iVum”ber of Jobs Processed — This analysis will indicatethe type of programmed that are used’ most frequently and those whichshould be most efficient when running.
4.5.3 Mean Time to Process Jobs – This characteristic will show howlong jobs have taken to run; this will include any queuing time. It maybe necessary to adjust a priority structure to the system to achieve asatisfactory balance or to increase the speed of some common peripherals.
4.5.4 Mean Time to Prepare Jobs — This time will indicate how wellthe manual part of the system is functioning. If long delays are occurringthen additional staff or more eflicient facilities may be required.
4.5.5 Time Taken to Archive — If this time shows excessive increasethen more effective means should be found as the integrity of the systemcould “be at risk.
4.5.6 Unsuccessful 1.4ttempts to A@uire Peripherals — Use should bemade of any in-built fault logs; these logs should record the number oftimes the operating system has been unsuccessful in acquiring aperipheral. This number could indicate excessive loading or some faultyequipment.
4.5.7 Spare Storage — The rate of consumption-of spare storage sincethe last audit should be considered. If-excessive, this consumption ratecould indicate either the need to .prpvlde additional capacity, or a poorallocation/collecting algorithm wlthm the system.
4.5.8 Amount of Useful Spare Processor Time Remaining — This para-meter should. be assessed accurately as it may l~mit the efficiency of thesystem. If it M Inadequate and process completion times are rising, thenadditional cap.ac]ty should be procured or more econonuc programmedproduced.
21
.r“ ,
,/..
-. .
/’
4;5.9” System @awn: Tfmp %infle .Lqyt ~wit = ~~x,a~lkmqqglg ,$Q ty?p-sider the amount of down $ime sia~ lks lq@ ~ygljf pg~ #~~p~ ,t~ ~~gr,mine any system@ie .pa$twm.
+l,~q~ ~scivpmcy /?f?pQ@ arid Chgqge Note# -“In a@itiorI toexamming the’ data related to performance charadei%$i$q, -“~$e auditshould consider discrepancies notified by the user and assoeaatcd eemee-tive action. These considerations rqq~ke @M the g~r ~@q@@@n hasset .up procedures far dealing withi
a) discrepancy repewting, and
b) c~ange ccwtrol.
The audit should try to correlate the gwrtufbqtiqq in jW$fOFpIaQ@
chariwterist jcs with @@ciency reports. If correlation cannot be establi-shed for particular deficlenci.es, the auditor should consider if additionalcharacteristics need to be measured. When piaqning an 8ud@, congiidera,tion should be given to changes implemented in “response to deficiencyreports to da$ermiop wh@ effect this aqtjoq had on the perforpaqce ofthe system.
4.6 User 13we-+m
4,6.Q Ge~rral -4 ~o@ $yst~m is one that me.gls the requiremerits ofthe user, tp ~h~ ywr’$ $f@#actiop. The @c@ fques recommended in thisstandard can he qs~ to eqsuro that the sta%el requ@ment? of the usercontinue to be met. ‘However, as user awareness of the system and tom=puter systems in ge~~~gl immases, ~xpectations may change and dissa#is-facticm set in. A @ssflfjs66d user might disre~rd the sy~te~, use it care-lessly or misuse it and the effect wdl be a detrimental c~p~~ 1P pytjternperformance.
4,fi,0il The User ~urvey, as a $echqique qf approa+ir+$ user$, .eanbe ccmpar.ed to an oplnlon poll. Thq que@iens sh@uld W @@ally ~cn,strutted so as to be prgpise and not mis!ea@ing. TIw uo~ler ,of *Qsurvey should take care that the questions th!wehes do mat w $Qdissatisfaction by suggesting the system could be bettered. Theqtw@iQpsshe~l~ aim be within t~p ,~o~e and ,qo~pr~p’sion of thg user.
4+.0.2 ‘The rnethQds by which tl!w SWWBY WY b WMhflk$. 5h@d
7$be considered. A regular question~m, di@ibutg@ by ~~ iqtgfmemo, to be filled in and returned, twll ptmvde @Iri%~@$ %1.a tf~ Fb~sis~ I.w@I@ all users will respo~~. An q~ppriexlqe~ mterwewer may$@ a hetier xpsP~~sq a@ be @le, Q qssg$ the gsers’ answqrs #mt@ie isvery timq W~Sg~l~g,
4.6.3 ~OktbYl of Ftrets — A +mor.sury~ oan b~ M 4P ,gmg?changes in requirements and the effect of changes made b~@ ~
22
- - - - -.:~
!
1
!
,
.!
syst$m I@ng mo~i$ored and other sys$erns whi@ may @ee$ it, forI example:~
.aj volume of input,
b) frequency of use,!
c) increased work load caused by a change to some other system,and
d) rmponse time.
4.6.2 Determination of the Level of Satisfaction — Although theperformance of a system may chaage, it is also neeessary to dekrminethe importance of the change to the user. A system may run unchangedin all respects other than the level of expectation of the user, or withcontinual use he may come across operating difficulties which will causedissatisfaction or annoyance, for example:
a) the ‘abort’ “button is next to ‘accept’ button on the key bcmd,
b) he has seen a differertt design of VDU, and
c) the space on a report is not sufllcient for additional handwrittennotes.
4.6.3 Prevention of Dissatisfaction — The user survey may preventpsychological dissatisfaction growing, as it is evidence of the designer’scontinuing concern for the user and that the user is the driving forcebehind the system not vice-versa.
5. PERFORMANCE MOIWTORING REPORTING
5.0 General
5.0.1 This secticm makes recommendations to assist the user to estab-lish a system of performance monitoring reporting that enables changesin performance detected by monitoring techniques to be analysed andpresented in a meaningful fashion.
5.0.2 When deciding upon the scope of a reporting system it is neces-sary to be dear about its purpose and value so that true cost effectivenessis achieved. Inadequate reporting can fail to fullll its purpose while @superfluity of records and reports can be both costly arid iead to impor-tant trends being overlooked among a mass of va]geless info~~atjon.
5.0.3 It is important that mrmagement spe$ilks cle.arjy @e form inwhich performance? jnfqrrnatfon shotild be presented.
5.1 Performance Criteria
5.1.0 Genera! — An e$6@ial found@@ Pf MM%FIBMMMtrwikming
23
________,..——. ———
.,...
.
,.,
IS :11289-1985
reporting is a schedule of the criteria by which the system to be moni-tored will be assessed, together with an acceptable value ( or range ofvalues ) for each criterion. Because system development and operationis often a dynamic process, this schedule will contain a mixture oforigina”l criteria and revised and additional criteria.
5.1.1 Original Criteria — These criteria should be obtainable fromthe system specification described in IS : 11290-1985* and should be setout in the form of a schedule of criteria to be met by the system beingmonitored.
5.1.2 Revised Criteria — Revised criteria can arise from:
a) amendments made during system development and documentedin accordance with relevant standard ( IS : 11290-1985* ).
b) temporary or permanent concessions recorded on the testsummary described in relevant standard ( IS : 11291- 1985t ).
c) changes made subsequent to the handover of the system.
5.2 Evaluation of Performance Measurement Data
5.2.1 From the data produced by the chosen monitoring techniques,measures of achievement can be recorded for comparison with thescheduled performance criteria. In assessing the significance of thesecomparisons it is important to take account ofi
a) any assumptions made with respect to the conditions in whichthe monitoring took place,
b) problems -encountered in applying the monitoring techniques,and
c) how accurately the techniques applied actually measure the criteriabeing monitored,
5.2.2 It is also important to ensure that all significant measures andcomparisons are truly comparing like with like.
Similarly careful consideration should be given to the period oftime or number of occurrences which should be included in thesecomparisons and records so that, for example, periodic variations orlong term trends do not escape attention.
5.3 Performance Reporting
5.3.1 Care should be taken in determining the precise form in whichperformance reports should be presented so that their practical or
*CO& of practice for documentation of computer based system.
tCode of practice for testing of computer based system.
24.
,, .
.
,,.
,.
.
., -1\
,,
,,
,--,.,,
IS :11289-1985
economic significance is made perfectly clear to the recipients. Thenature .of the contents may suggest whether they should take the formof, for example:
a) exception reports;
b) statistical analyses and summaries; and
c) trend reports ( for example, charts or graphs ).
5.3.2 In additional to the fcrm of presentation, the time at whichreports are submitted ( in relation to the comparisons which they con-tain ) can be of importance for management control. Considerationshould be given to the cost effectiveness of having reports made:
a) immediately a change in performance is detected;
b) on trends reaching predetermined levels or values; and
c) periodically ( for example, daily, weekly, monthly, annually).
5.3.3 Consideration should also be given to the possibility that theapplication of alternative reporting criteria could assist managementdecision making. For example, a system loading report may be madeat set periods but, in addition, immediately a load exceeding a givenvalue is detected.
5.3.4 Because computer-based systems tend to be dynamic in nature,it is essential to ensure that the monitoring and reporting is reviewedon a regular basis to ensure that it is at all times:
a) relevant to the current environment,
b) presented in the most suitable form,
c) timely in its submission, and
,,,.
,*
d) cost justifiable.
25
..M,, . . .. ..” .-,,. .~,,.,,.,...,” . ~., . !..—
.
.,-’.
,.,
,_-..
