Quality Assurance - Web viewExpand the word COBIT? ... Such a system formed the basis for lessons for top management in Japan in 1950 and in subsequent ... Slogans

IT QA AND AUDIT - Iyengar Sir

Table of ContentsQuality Assurance......................................................................................................................................................................3

1. Draw the following quality process improvement tools and give two points on how each of these tool help in process improvement............................................................................................................................................................................. 3

2. What is QMS? Explain the elements (clauses) of QMS process model using ISO 9001:2008? Map these to the software development phases or activities. Identify at least two major challenges that arise in each of these phases and show how QMS help us meet these challenges?........................................................................................................................................7

3. Name the six maturity levels in CMMi? Explain in brief these levels and in your own words describe any “generic goals” and “specific goals” in a software development organization that is ranked “Performed” as its capability maturity level.......9

4. Name the six maturity levels in CMMi? Explain in brief these levels and in your own words describe any “generic goals” and “specific goals” in a software development organization that is ranked “Managed” as its capability maturity level.........9

5. Name the eight quality management principles and explain each one of the following with suitable example:............12

6. What is your understanding about the terms “Quality Assurance” and “Quality Control”? Name any two measures for each for QA and QC. Explain the same with suitable example.................................................................................................15

7. Explain briefly any 10 of 14 management principles of Deming and relate each principle to an information management or software development industry.....................................................................................................................16

8. “Do not have unrealistic targets” OR “Eliminate quotas and numerical targets”. Explain with suitable example in real time to demonstrate how this principle is to be implemented and practiced.........................................................................19

9. What is the meaning of the term (i) measure and (ii) metrics? Name and explain at least four software quality metrics? Name any two software attributes that are normally measured? List and explain any two metrics for each of these two software attributes. (2 metrics for 2 attributes = 4 measurements)........................................................................................19

10. Write a brief note on “Benchmarking”. (Points expected: What is it? Who is it for? How to implement? And Benefit realization)...............................................................................................................................................................................21

Information System Audit and Control Practices.....................................................................................................................27

11. What do you understand by the term Information Systems Audit? You have been asked to conduct an IS audit for 3 locations of JB Technologies Ltd, a software development company, at multi-cities within India (17 locations), United Kingdom (3 locations), and the United States of America (13 locations).What is the basis on which you will choose your 3 locations? What areas (at least 2 main and 2 support functions) of organization’s business practices will need to be covered? List at least 3 things that you will cover in each of these business practices. Explain briefly justifying your stand taken above............................................................................................................................................................................. 27

12. Describe briefly with suitable examples as to how introduction of Information System Audit can improve the organization’s:..........................................................................................................................................................................29

13. Describe briefly the four major activities of Information System Audit Process. (Planning, Conducting of Audit, Reporting of Audit findings, and Follow-up)............................................................................................................................29

14. Explain the terms: (i) IT Service Management (ITSM) and (ii) IT Service Management System (ITSMS)? Describe briefly all 13 IT service processes of ISO 20000-1:2011 and map the same with software service activities...........................32

15. How does control of “Change” and “Configuration” help in controlling Quality of Services in Business Application Releases?................................................................................................................................................................................. 36

16. Explain the terms: (i) Information Security (IS), (ii) Information Security Management (ISM) and Information Security Management System (ISMS)? Describe briefly the PDCA cycle of ISO 27001:2005 and its 11 control areas or domains........36

IT QA & Control Audit Question Bank

17. Explain the terms: (i) Business Continuity and (ii) Business Continuity Management System (BCMS)? Describe the five elements (stages) of a BCM program management and map them for an Information management or software development company............................................................................................................................................................40

18. Explain the terms: (i) Control and (ii) Control Objectives. Expand the word COBIT? Name the four major areas of COBIT and explain your understanding on each one of them..................................................................................................41

19. Name and explain in brief (one or two sentences) the five principles of COBIT 5. Name and explain the seven enterprise enablers used in COBIT 5........................................................................................................................................43

20. What is your understanding about (i) risk and (ii) risk management with respect to an IT enterprise? Explain how the terms threat, impact, etc. relate to risk. Explain briefly the activities that are carried out during risk identification, risk estimation, risk evaluation and risk treatment in the overall risk management process.........................................................44

21. What constitute an eCommerce activity(ies)? What will you audit in an eCommerce environment? Describe the content of an audit report with respect to the objective of the audit, outcome or audit findings for an eCommerce business.

47

Page 2


Quality Assurance

1. Draw the following quality process improvement tools and give two points on howeach of these tool help in process improvement.

(a) Flowchart

A flow chart is a graphical or symbolic representation of a process. Each step is represented by a different symbol and these symbols are linked together with arrows showing the process flow direction. Flowcharts are used in analyzing, designing, documenting or managing a process or program in various fields. Flowcharts are used in designing and documenting complex processes or programs. Like other types of diagrams, they help visualize what is going on and thereby help the people to understand a process, and perhaps also find flaws, bottlenecks, and other less-obvious features within it.

Basic Flowchart SymbolsFor most flowcharts, these five basic symbols are all you will need.

(b) Check sheetThe check sheet is a form or document used to collect data in real time at the location where the data is generated. The data it captures can be quantitative or qualitative. When the information is quantitative, the check sheet is sometimes called a tally sheet. The defining characteristic of a check sheet is that data are recorded by making marks ("checks") on it. A typical check sheet is divided into regions, and marks made in different regions have different significance. Data are read by observing the location and number of marks on the sheet.

Page 3

http://www.smartdraw.com/specials/images/examples/flowchart-symbols.gif


The most straightforward check sheet is simply to make a list of items that you expect will appear in a process and to mark a check beside each item when it does appear. This type of data collection can be used for almost anything, from checking off the occurrence of particular types of defects to the counting of expected items (e.g., the number of times the telephone rings before being answered).

(c) HistogramA histogram is a graphical representation of the distribution of data. It is a display of statistical information that uses rectangles to show the frequency of data items in successive numerical intervals of equal size. In the most common form of histogram, the independent variable is plotted along the horizontal axis and the dependent variable is plotted along the vertical axis. The data appears as colored or shaded rectangles of variable area.

The illustration, below, is a histogram showing the results of a final exam given to a hypothetical class of students. Each score range is denoted by a bar of a certain color. Conclusions might also be drawn concerning the improvement or decline of the professor's teaching ability with the passage of time. If this histogram were compared with those of other classes in the same semester who had received the same final exam but who had taken the course from different professors, one might draw conclusions about the relative competence of the professors.

(d) Pareto ChartA Pareto chart is a bar graph. The lengths of the bars represent frequency or cost (time or money), and are arranged with longest bars on the left and the shortest to the right. In this way the chart visually depicts which situations are more significant. The bars are arranged in descending order of height from left to right. This means the categories represented by the tall bars on the left are relatively more significant than those on the

Page 4


right. The chart gets its name from the Pareto Principle, which postulates that 80 percent of the trouble comes from 20 percent of the problems.

(e) Scatter diagramA scatter diagram is a tool for analyzing relationships between two variables. One variable is plotted on the horizontal axis and the other is plotted on the vertical axis. The pattern of their intersecting points can graphically show relationship patterns. Most often a scatter diagram is used to prove or disprove cause-and-effect relationships.

(f) Control chart

The control chart is a graph used to study how a process changes over time. Data are plotted in time order. A control chart always has a central line for the average, an upper line for the upper control limit and a lower line for the lower control limit. These lines are determined from historical data. By comparing current data to these lines, you can draw conclusions about whether the process variation is consistent (in control) or is unpredictable (out of control, affected by special causes of variation).

It is used to determine whether or not a process is stable or has predictable performance. Typically, control charts identify upper and lower control limits to determine the acceptable range of test results. Control charts commonly have three types of lines: 1. Upper and lower specification limits2. Upper and lower control limits3. Planned or goal value

Control charts illustrate how a process behaves over time and defines the acceptable range of results. When a process is outside the acceptable limits, the process is adjusted. Control charts can be used for both project and product life cycle processes. For example, for project processes a control chart can be used to determine whether cost variances or schedule variances are outside of acceptable limits.

(g) Cause-and-effect diagramCause-and-effect diagram is also known as the Fishbone Diagram or the Ishikawa Diagram. Common uses of the Ishikawa diagram are product design and quality defect prevention, to identify potential factors causing an overall effect. Each cause or reason for imperfection is a source of variation. It is a tool used for systematically identifying and presenting all the possible causes of a particular problem in graphical format. The possible causes are presented at various levels of detail in connected branches, with the level of detail increasing as the

Page 5


branch goes outward, i.e., an outer branch is a cause of the inner branch it is attached to. Thus, the outermost branches usually indicate the root causes of the problem.

The Ishikawa Diagram resembles a fishbone (hence the alternative name “Fishbone Diagram”) – it has a box (the ‘fish head’) that contains the statement of the problem at one end of the diagram. From this box originates the main branch (the ‘fish spine’) of the diagram. Sticking out of this main branch are major branches that categorize the causes according to their nature. In semiconductor manufacturing, 4 major branches are often used by beginners, referred to as the ’4 M’s', corresponding to ‘Man’, ‘Machine’, ‘Materials’, and ‘Methods’. Sometimes 5 branches are used (’5 M’s'), with the fifth branch standing for ‘Measurement’, or even ‘M-ironmen.’ These ‘M’s’ or problem cause categories are used to classify each cause identified for easier analysis of data. Of course, one is not constrained to use these categories in fishbone diagram. Experienced users of the diagram add more branches and/or use different categories, depending on what would be more effective in dealing with the problem.

(h) Run chartWe have discussed the histogram and Pareto chart. Think of both of these tools as similar to a camera where a snapshot of the process has been taken. But the run chart is similar to a camcorder, recording some process element over time.

A run chart is a line graph that shows data points over time. Run charts are helpful in identifying trends and predicting future performance. Run charts are similar to control charts, plotting data results over time; however there are no defined control limits.

ExampleA control chart may be used for a pharmaceutical company that is testing a new pain medication. The drug must stay effective in the system for a minimum of three hours but last no more than five hours, to prevent accidental overdose. The mean time or goal efficacy duration would be four hours, with three hours the lower control limit and five hours the upper control limit.

Page 6


A run chart may be used to plot the temperature within the manufacturing plan every day for a month to determine a trend.

2. What is QMS? Explain the elements (clauses) of QMS process model using ISO 9001:2008? Map these to the software development phases or activities. Identify at least two major challenges that arise in each of these phases and show how QMS help us meet these challenges?

Quality can be defined as Based on customer’s perceptions of a product/service’s design and how well the design matches the original

specifications. The ability of a product/service to satisfy stated or implied needs. Achieved by conforming to established requirements within an organization.

What Is a Quality Management System (QMS)?A quality management system is a management technique used to communicate to employees what is required to produce the desired quality of products and services and to influence employee actions to complete tasks according to the quality specifications.

What Purpose Does a Quality Management System Serve? Establishes a vision for the employees. Sets standards for employees. Builds motivation within the company. Sets goals for employees. Helps fight the resistance to change within organizations. Helps direct the corporate culture.

ISO 9001:2008 Quality Management System

Page 7

CUSTOMER

REQU I REMENT

SAT I S FAC T I ON

PRODUCT

CUSTOMER


Main Clauses

Quality Management System Management Responsibility

o Management Commitment Establish policy and objectives (SMART) Communicate importance of meeting Customer and regulatory Requirements Conducting Management Reviews Ensuring availability of resources

o Customer Focuso Responsibility, Authority and communication

Resources Managemento Provision of Resources (in a timely manner)o Human Resources

Competence Awareness Training

o Infrastructureo Work Environment

Product Realizationo Customer-related processes

Determination of requirements

Page 8


Review of requirements Customer communication

o Design and Development o Purchasing

Vendor Selection & evaluation Purchasing information Verification of purchased product

Measurement, Analysis and Improvemento Control of Monitoring and Measuring Devices (Calibration)o Monitoring and Measurement

Customer Satisfaction Internal Audit

o Analysis of Data Customer Satisfaction Conformity to product requirements

o Supplierso Improvemento Continual improvement through policy, objectives, audit results, analysisof data, corrective/preventive

actions and management reviews

3. Name the six maturity levels in CMMi? Explain in brief these levels and in your ownwords describe any “generic goals” and “specific goals” in a software developmentorganization that is ranked “Performed” as its capability maturity level.

4. Name the six maturity levels in CMMi? Explain in brief these levels and in your ownwords describe any “generic goals” and “specific goals” in a software developmentorganization that is ranked “Managed” as its capability maturity level.

Capability Maturity Model Integration (CMMI) is a process improvement training and appraisal program and service administered and marketed by Carnegie Mellon University and required by many DOD and U.S. Government contracts, especially in software development. Carnegie Mellon University claims CMMI can be used to guide process improvement across a project, division, or an entire organization. Under the CMMI methodology, processes are rated according to their maturity levels, which are defined as:

A maturity level is a well-defined evolutionary plateau toward achieving a mature software process. Each maturity level provides a layer in the foundation for continuous process improvement.

In CMMI models with a staged representation, there are five maturity levels designated by the numbers 1 through 51. Initial2. Managed3. Defined4. Quantitatively Managed5. Optimizing

Page 9


Maturity levels consist of a predefined set of process areas. The maturity levels are measured by the achievement of the specific and generic goals that apply to each predefined set of process areas. The following sections describe the characteristics of each maturity level in detail.

Maturity Level 1 - Initial Processes are usually ad hoc and chaotic. The organization usually does not provide a stable

environment. Success in these organizations depends on the competence and heroics of the people in the organization and not on the use of proven processes.

Organizations often produce products and services that work; however, they frequently exceed the budget and schedule of their projects.

Organizations are characterized by a tendency to over commit, abandon processes in the time of crisis, and not be able to repeat their past successes.

Maturity Level 2 - Managed An organization has achieved all the specific and generic goals of the maturity level 2 process areas. In

other words, the projects of the organization have ensured that requirements are managed and that processes are planned performed, measured, and controlled.

The process discipline reflected by maturity level 2 helps to ensure that existing practices are retained during times of stress. When these practices are in place, projects are performed and managed according to their documented plans.

Requirements, processes, work products, and services are managed. The status of the work products and the delivery of services are visible to management at defined points.

Commitments are established among relevant stakeholders and are revised as needed. Work products are reviewed with stakeholders and are controlled.

The work products and services satisfy their specified requirements, standards, and objectives.

Maturity Level 3 - Defined An organization has achieved all the specific and generic goals of the process areas assigned to maturity

levels 2 and 3. Processes are well characterized and understood, and are described in standards, procedures, tools, and

methods.

Page 10


A critical distinction between level 2 and level 3 is the scope of standards, process descriptions, and procedures. At level 2, the standards, process descriptions, and procedures may be quite different in each specific instance of the process (for example, on a particular project). At maturity level 3, the standards, process descriptions, and procedures for a project are tailored from the organization's set of standard processes to suit a particular project or organizational unit. The organization's set of standard processes includes the processes addressed at maturity level 2 and maturity level 3. As a result, the processes that are performed across the organization are consistent except for the differences allowed by the tailoring guidelines.

Processes are typically described in more detail and more rigorously than at maturity level 2. At maturity level 3, processes are managed more proactively using an understanding of the interrelationships of the process activities and detailed measures of the process, its work products, and its services.

Maturity Level 4 - Quantitatively Managed An organization has achieved all the specific goals of the process areas assigned to maturity levels 2, 3,

and 4 and the generic goals assigned to maturity levels 2 and 3. At level 4 Sub processes are selected that significantly contribute to overall process performance. These

selected sub processes are controlled using statistical and other quantitative techniques. Quantitative objectives for quality and process performance are established and used as criteria in

managing processes. Quantitative objectives are based on the needs of the customer, end users, organization, and process implementers. Quality and process performances are understood in statistical terms and are managed throughout the life of the processes.

Special causes of process variation are identified and, where appropriate, the sources of special causes are corrected to prevent future occurrences.

Quality and process performance measures are incorporated into the organization’s measurement repository to support fact-based decision making in the future.

A critical distinction between maturity level 3 and maturity level 4 is the predictability of process performance. At maturity level 4, the performance of processes is controlled using statistical and other quantitative techniques, and is quantitatively predictable. At maturity level 3, processes are only qualitatively predictable.

Maturity Level 5 - Optimizing At maturity level 5, an organization has achieved all the specific goals of the process areas assigned to

maturity levels 2, 3, 4, and 5 and the generic goals assigned to maturity levels 2 and 3. Processes are continually improved based on a quantitative understanding of the common causes of

variation inherent in processes. Level 5 focuses on continually improving process performance through both incremental and innovative

technological improvements. Quantitative process-improvement objectives for the organization are established, continually revised to

reflect changing business objectives, and used as criteria in managing process improvement. The effects of deployed process improvements are measured and evaluated against the quantitative

process-improvement objectives. Both the defined processes and the organization's set of standard processes are targets of measurable improvement activities.

Optimizing processes that are agile and innovative depends on the participation of an empowered workforce aligned with the business values and objectives of the organization. The organization's ability to rapidly respond to changes and opportunities is enhanced by finding ways to accelerate and share learning. Improvement of the processes is inherently part of everybody's role, resulting in a cycle of continual improvement.

A critical distinction between maturity level 4 and maturity level 5 is the type of process variation addressed. At maturity level 4, processes are concerned with addressing special causes of process variation and providing statistical predictability of the results. Though processes may produce

Page 11


predictable results, the results may be insufficient to achieve the established objectives. At maturity level 5, processes are concerned with addressing common causes of process variation and changing the process (that is, shifting the mean of the process performance) to improve process performance (while maintaining statistical predictability) to achieve the established quantitative process-improvement objectives.

5. Name the eight quality management principles and explain each one of the following with suitable example:

a. Fact based decision makingb. Mutually beneficial supplier Relationships

Definition of Quality Management PrincipleA quality management principle is a comprehensive and fundamental rule / belief, for leading and operating an organization, aimed at continually improving performance over the long term by focusing on customers while addressing the needs of all other stake holdersA principle is a fundamental truth or law and therefore quality management principles are the fundamental truth or laws that form the basis of quality management.Customer focused organization

Organizations depend on their customers and therefore should understand current and future customer needs, should meet customer requirements and strive to exceed customer expectations

Leadership Leaders establish unity of purpose and direction. They should create and maintain the internal environment in which people can become fully involved in achieving the organization's objectives

Involvement of people People at all levels are the essence of an organization and their full involvement enables their abilities to be used for the organization's benefit.

Process approach A desired result is achieved more efficiently when activities and related resources are managed as a process.

System approach to management

Identifying, understanding and managing a system of interrelated processes as a system contributes to the organization’s effectiveness and efficiency in achieving its objectives.

Continual improvement Continual improvement of the organization's overall performance should be a permanent objective of the organization.

Factual approach to decision making

Effective decisions are based on the analysis of data and information.

Mutually beneficial supplier relationships

An organization and its suppliers are interdependent and a mutually beneficial relationship enhances the ability of both to create value.

Customer focused organizationSteps in applying this principle: Understand customer needs and expectations for products, delivery, price, dependability, etc. Ensure a balanced approach among customers and other stake holders (owners, people, suppliers, local

communities and society at large) needs and expectations. Communicate these needs and expectations throughout the organization. Measure customer satisfaction & act on results, and Manage customer relationships.

Key Benefits Increased revenue and market share obtained through flexible and fast responses to market opportunities

Page 12


Increased effectiveness in the use of the organization’s resources to enhance customer satisfaction Improved customer loyalty leading to repeat business.

LeadershipSteps in applying this principle: Be proactive and lead by example. Understand and respond to changes in the external environment. Consider the needs of all stake holders including customers, owners, people, suppliers, local communities and

society at large. Establish a clear vision of the organization’s future. Establish shared values and ethical role models at all levels of the organization. Build trust and eliminate fear. Provide people with the required resources and freedom to act with responsibility and accountability. Inspire, encourage and recognize people's contributions. Promote open and honest communication. Educate, train and coach people. Set challenging goals and targets, and Implement a strategy to achieve these goals and targets.

Key Benefits: People will understand and be motivated towards the organization’s goals and objectives Activities are evaluated, aligned and implemented in a unified way Miscommunication between levels of an organization will be minimized.

Involvement of peopleSteps in applying this principle: Accept ownership and responsibility to solve problems. Actively seek opportunities to make improvements, and enhance competencies, knowledge and experience. Freely share knowledge & experience in teams. Focus on the creation of value for customers. Be innovative in furthering the organization’s objectives. Improve the way of representing the organization to customers, local communities and society at large. Help people derive satisfaction from their work, and Make people enthusiastic and proud to be part of the organization.

Key Benefits: Motivated, committed and involved people within the organization Innovation and creativity in furthering the organization’s objectives People being accountable for their own performance People eager to participate in and contribute to

continual improvement.

Process approachSteps in applying this principle: Define the process to achieve the desired result. Identify and measure the inputs and outputs of the process. Identify the interfaces of the process with the functions of the organisation. Evaluate possible risks, consequences and impacts of processes on customers, suppliers and other stake

holders of the process. Establish clear responsibility, authority, and accountability for managing the process. Identify internal and external customers, suppliers and other stake holders of the process, and

Page 13


When designing processes, consider process steps, activities, flows, control measures, training needs, equipment, methods, information, materials and other resources to achieve the desired result.

Key Benefits: Lower costs and shorter cycle times through effective use of resources Improved, consistent and predictable results Focused and prioritized improvement opportunities.

System approach to managementSteps in applying this principle: Define the system by identifying or developing the processes that affect a given objective. Structure the system to achieve the objective in the most efficient way. Understand the interdependencies among the processes of the system. Continually improve the system through measurement and evaluation, and Estimate the resource requirements and establish resource constraints prior to action.

Key Benefits: Integration and alignment of the processes that will best achieve the desired results Ability to focus effort on the key processes Providing confidence to interested parties as to the consistency, effectiveness and efficiency of the

organization.

Continual improvementSteps in applying this principle: Make continual improvement of products, processes and systems an objective for every individual in the

organization. Apply the basic improvement concepts of incremental improvement and breakthrough improvement. Use periodic assessments against established criteria of excellence to identify areas for potential

improvement. Continually improve the efficiency and effectiveness of all processes. Promote prevention based activities. Provide every member of the organization with appropriate education and training, on the methods and tools

of continual improvement such as the Plan-Do-Check-Act cycle , problem solving , process re-engineering, and process innovation.

Establish measures and goals to guide and track improvements, and Recognize improvements.

Key Benefits: Performance advantage through improved organizational capabilities Alignment of improvement activities at all levels to an organization’s strategic intent Flexibility to react quickly to opportunities.

Factual approach to decision makingSteps in applying this principle: Take measurements and collect data and information relevant to the objective. Ensure that the data and information are sufficiently accurate, reliable and accessible. Analyze the data and information using valid methods. Understand the value of appropriate statistical techniques, and Make decisions and take action based on the results of logical analysis balanced with experience and intuition.

Page 14


Key Benefits: Informed decisions An increased ability to demonstrate the effectiveness of past decisions through reference to factual records Increased ability to review, challenge and change opinions and decisions.Mutually beneficial supplier relationshipsSteps in applying this principle: Identify and select key suppliers. Establish supplier relationships that balance short-term gains with long-term considerations for the

organization and society at large. Create clear and open communications. Initiate joint development and improvement of products and processes. Jointly establish a clear understanding of customers' needs. Share information and future plans, and Recognize supplier improvements and achievements.

Key Benefits: Increased ability to create value for both parties Flexibility and speed of joint responses to changing market or customer needs and expectations Optimization of costs and resources.

6. What is your understanding about the terms “Quality Assurance” and “Quality Control”? Name any two measures for each for QA and QC. Explain the same with suitable example.

Quality control (QC) is a procedure or set of procedures intended to ensure that a manufactured product or performed service adheres to a defined set of quality criteria or meets the requirements of the client or customer. QC is similar to, but not identical with, quality assurance (QA)

QA is defined as a procedure or set of procedures intended to ensure that a product or service under development (before work is complete, as opposed to afterwards) meets specified requirements. QA is sometimes expressed together with QC as a single expression, quality assurance and control (QA/QC).

Measures for Quality Assurance

Benchmarking and Process Improvement

Benchmarking is practical with minimal data, namely the time, total effort and software size for main development phase. Defects are an optional extra. This data is collected for recently completed projects. Purchasing requests the same data to build benchmark measures of supplier performance and uses the results to negotiate current and future developments. SQA verify this data is being collected and is used to build the benchmark database.

As developments complete a review examines the detailed monthly progress data collected by the Software Control Office (SCO). SQA participates in the review to verify the SCO has assembled the complete history for each development. The final data is added to the benchmark database of projects. Over time this growing database provides concrete evidence of development process productivity and quality improvement.

Page 15


A regular procedure quantifies improvement benefits. At intervals, say every 6 months, a report is made showing benefits from recent projects through initiatives such as CMMI. This includes calculating the Return on Investment (ROI) based on productivity improvement and investments made to improve.

The Estimating and Risk Assessment ProcessIn a development group SQA is performed on the documented estimating procedure to check input data is used that quantifies:• The software product size and uncertainty• The development process productivity• The development constraints: time, effort, staff, and reliability• The risk levels for each constraint

Software acquisition requires the supplier to provide this data using a formal questionnaire. Software size is quantified using the estimated size range. The range reflects specification uncertainty that reduces as progress is made through the feasibility and specification phases. Greater detail is practical as feature specifications are refined. Each software module is estimated in terms of the smallest, most likely and largest size. This size range uses the most practical sizing units such as logical input statements, function points or objects

The time, effort, resources, costs and reliability constraints for the development are risk assessed taking in to account the quantified uncertainties such as the size range. Each constraint is associated with a risk level and estimates are evaluated against these risk levels.

Frequently it is found that specific constraints cannot be met since the risk is too high. The estimating procedure evaluates alternatives, each of which is logged and documented. This may mean allowing additional time, adding staff and/or reducing features (size). The alternative “What If” estimates document how the final baseline plan is determined, risk assessed and agreed.

7. Explain briefly any 10 of 14 management principles of Deming and relate each principle to an information management or software development industry.

W. Edwards Deming’s 14 points are the basis for transformation of industry. Adoption and action on the 14 points are a signal that the management intends to stay in business and aims to protect investors and jobs. Such a system formed the basis for lessons for top management in Japan in 1950 and in subsequent years.The 14 points apply anywhere, to small organizations as well as to large ones, to the service industry as well as to manufacturing. They equally apply to any division within a company and to its suppliers. Deming’s fourteen points of management approach provide guidelines for implementing the TQM concept. These fourteen points can be applied to managing software development processes.

1) Create constancy of purpose for improvement of product and service.Software development process traditionally ends when the completed system is handed over to the support group and put into production mode. Under the TQM culture, there is no finish line for the development team. Maybe there is a shift of focus from one project to another. The development team should be responsible for what they delivered, not the support group. Any quality problem occurs during the production should be addressed to the development team. Management must [Zultner, 1988]: • Establish operational definitions for each step in the software development process. • Define what is meant by “service to the customer.” • Define standards of development, maintenance, and service for the next year and five years ahead. • Define the internal and external customer. • Develop ways to provide better systems and services in less time, using fewer resources.

Page 16


• Invest in tools and techniques for better software development.

2) Adopt the new philosophy of total quality. Quality is everyone’s business. Not just the worker, management is part of the quality team. Under the TQM culture, quality comes first and everyone must join in. Corporate management, from top to bottom, must embrace the TQM concept and clearly communicate their support of this concept to all members in the software development team.

3) Cease dependence on mass inspection to achieve quality.Quality is built in, not added on. It is better to prevent errors in code, rather than reworking the code to remove the errors. Inspection or testing cannot prevent errors from happening, only experience and knowledge can. Management must install programs to continually improve software development processes. Examples of such programs are job training and job incentive programs.

4) End the practice of awarding business based on price tag alone.Many software organizations today are outsourcing their projects to subcontractors. It is important not to award a software contract based on price tag alone. Quality is more important than the difference in costs. Low quality in the long run will result in high total cost. It is better to create long-term relationship with a few loyal and trustworthy suppliers who can produce quality code for your system.

5) Improve constantly and forever the systems of production and service.System development processes must be constantly improved by introducing new and working methodology, paradigm, standards, practices, techniques, tools, policies, and procedures. All these require the organization to constantly keep tracking the best practice in the field of management information system (MIS)—the so-called learning organization. Each individual staff member is required to improve oneself by updating or even expanding one’s skill set.

6) Institute training on the job. To build quality into the software, the development team must have appropriate experience and knowledge. On-the-job training program is an effective means of obtaining such experience and knowledge. In the broadest sense, all MIS staff members must know what their jobs entail and how to do their work. Management must assess the skill level of an employee before he or she is assigned to a software project. Different skill levels can play different roles and assume different responsibilities in a project.

7) Institute leadership. Management must lead, not punish. It is manager's job to help MIS staff do a better job and create a better system. Project managers must be trained in basic interpersonal and analytical skills. They must have a solid understanding of statistical process control. They should know that in any software development team whose performance is in statistical control, half of them would always be below average. They should focus on those members whose performance is out of statistical control.

8) Drive out fear of job insecurity. Employees must feel secure before they are willing to ask questions, make suggestions, or even expose their weaknesses by asking for help. The policy of long-term employment could easily drive out the fear of job insecurity. Moreover, any MIS staff whose performance is out of statistical control should be offered help in retraining or reassignment. However, if one consistently rejects helps from one's co-workers or supervisors, a layoff may be the last resort.

9) Break down barriers between departments or staff areas. Software development requires collaborative effort between users and IS staff. For as long as we can remember, communication gap has been the major factor to many MIS implementation failures. Furthermore, today's

Page 17


business system projects would most likely involve different functional areas and require expertise in database processing, client-server computing, and network installation, etc. Therefore, open communication among functional areas and general knowledge across disciplines are necessary for a successful system implementation. This requires appropriate education and training for team members to change their behavior and improve their knowledge.

10) Eliminate slogans, exhortations, and targets for the workforce. Slogans do not build quality systems. MIS management should not ask for impossible target or schedule, or unrealistic level of productivity. Instead, they should post their progress in responding to suggestions and in helping the staff improve quality. Let the employees put up their own signs and slogans [Zultner, 1988].

11) Eliminate numerical quotas, , and work standards. Quotas (such as metrics), goals (such as schedules), and work standards (such as unit times) address numbers, not quality. A software development project that causes haste and non-conformities accomplishes nothing and services no one. Let the project members put up their own goals. Managers should concentrate on helping people do a better job by reducing rework, errors, and waste. Everyone must work toward constant improvement, not the achievement of some arbitrary, short-term goals [Zultner, 1988].

12) Remove barriers to pride of workmanship. All people are motivated. They would like to make quality products. However, a good workmanship relies on good materials, good tools, good methods, and good timing. Poor materials, broken tools, ineffective methods, or belated schedule are all barriers to pride of workmanship and should be eliminated. Let the software development team put its group identity or team members' names on the software product to take the credit (or the responsibility) of their work.

13) Institute a vigorous program of education and retraining for everyone. On-the-job training is effective, but slow, for an employee to acquire skill set for a particular type of job. In today's MIS arena, technology is changing so fast that new skill set is needed for the same type of job in a short period of time. Management must set aside enough budgets to execute a generous education and retraining program for everyone to improve oneself. Under the TQM culture, all employees must know enough statistical method to understand the nature of variation, to manage the special causes of variation. Support for training employees to acquire necessary statistical method should be institutionalized.

14) Put everyone to work to accomplish the transformation. The TQM transformation is everyone's job. Everyone has a customer. Ask yourself who is the person receiving your work? All of us must identify our customers in order to determine precisely what our jobs are. Everyone belongs to a team, to work in the Plan-Do-Check-Act cycle, to address one or more specific issues, to find special causes detected by statistical signals. Moreover, we must put management to work. Only management can change the culture and environment that dominate any individual's performance. Management must agree on their meaning and on the direction to take. They must acknowledge their mistakes, if any, and have the courage to change. They must explain to a critical mass of people in the organization why change is necessary and that the change will involve everybody. Obviously, people must understand the Fourteen Points to know what to do and how to do it [Walton, 1986].

Total quality management is not only a philosophy of work but also an ethic of workers. It is coming from the wisdom and the teachings of many quality improvement gurus. It has helped many companies to improve quality of products and processes, and in turn, increase the productivity and the profitability. Any software organization that is planning to implement the TQM must have the critical mass of its employees embrace the TQM philosophy and methods before jumping onto the bandwagon. That is, all employees regardless of their ranks must fully understand (or be trained with) and internalize the TQM concept and tools. To increase the

Page 18


chance of success, a TQM-implementation project should start from the top management and unfold it downward to lower-level management and workers with a goal to benefit the critical mass of employees. Specifically, the goal is to improve the quality of work life for the employees through improving work conditions, work methods, work compensation, work relations, and providing the employees with opportunities for professional development. Only with this goal could we gain the full cooperation from the employees and bring about successful TQM implementation.

Source: http://www.cob.calpoly.edu/~eli/pdf/jqai-00.pdf

8. “Do not have unrealistic targets” OR “Eliminate quotas and numerical targets”. Explain with suitable example in real time to demonstrate how this principle is to be implemented and practiced.

9. What is the meaning of the term (i) measure and (ii) metrics? Name and explain at least four software quality metrics? Name any two software attributes that are normally measured? List and explain any two metrics for each of these two software attributes. (2 metrics for 2 attributes = 4 measurements)

Metrics" and "measurements" can be viewed as different entities. A measures is a numerical value assigned to an attribute according to defined criteria, for example, one's current temperature in degrees FahrenheitA metric is a mathematical set of relevant, quantifiable, attributes (measures) taken over time. In this example, the metric would be a table or graph of one's temperature taken over a period of time, which would be helpful for identifying trends.

Schedule Variance = (ActualDt-PlannedDt)/Est.Calender Days Defect Density = Total Defects Found /Size

o Low Defect Density need not be good o may indicate poor reviews

Defect removal Efficiencyo Pre-Delivery Defects/Total No. of Defectso Ideally 100% target could be 95%o Indicates what % of defects are fixed before delivery.

Review Effectiveness o (Defects found@Review)/tot No. of Defectso Indicates the quality of Reviews

Common software metrics include: Bugs per line of code Code coverage Cohesion Coupling Cyclomatic complexity Function point analysis Number of classes and interfaces Number of lines of customer requirements Order of growth Source lines of code

Page 19

http://www.cob.calpoly.edu/~eli/pdf/jqai-00.pdf


Robert Cecil Martin’s software package metrics

Software Quality Metrics focus on the process, project and product. By analyzing the metrics the organization can take corrective action to fix those areas in the process, project or product which are the cause of the software defects.

The de-facto definition of software quality consists of the two major attributes based on intrinsic product quality and the user acceptability. The software quality metric encapsulates the above two attributes, addressing the mean time to failure and defect density within the software components. Finally it assesses user requirements and acceptability of the software. The intrinsic quality of a software product is generally measured by the number of functional defects in the software, often referred to as bugs, or by testing the software in run time mode for inherent vulnerability to determine the software "crash" scenarios. In operational terms, the two metrics are often described by terms namely the defect density (rate) and mean time to failure (MTTF).

Although there are many measures of software quality, correctness, maintainability, integrity and usability provide useful insight.CorrectnessA program must operate correctly. Correctness is the degree to which the software performs the required functions accurately. One of the most common measures is Defects per KLOC. KLOC means thousands (Kilo) Of Lines of Code.) KLOC is a way of measuring the size of a computer program by counting the number of lines of source code a program has.MaintainabilityMaintainability is the ease with which a program can be correct if an error occurs. Since there is no direct way of measuring an indirect way has been used to measure this. MTTC (Mean time to change) is one such measure. It measures when an error is found, how much time it takes to analyze the change, design the modification, implement it and test itIntegrityThis measure the system’s ability to with stand attacks to its security. In order to measure integrity two additional parameters are threat and security need to be defined. Threat – probability that an attack of certain type will happen over a period of time. Security – probability that an attack of certain type will be removed over a period of time. Integrity = Summation [(1 - threat) X (1 - security)]UsabilityHow usable is your software application? This important characteristic of your application is measured in terms of the following characteristics:

Physical / Intellectual skill required to learn the system Time required becoming moderately efficient in the system. The net increase in productivity by use of the new system. Subjective assessment(usually in the form of questionnaire on the new system)

Measure Metrics1. Customer satisfaction index

Number of system enhancement requests per year Number of maintenance fix requests per year User friendliness: call volume to customer service hotline User friendliness: training time per new user Number of product recalls or fix releases (software vendors) Number of production re-runs (in-house information systems groups)

2. Delivered defect quantities

Normalized per function point (or per LOC) At product delivery (first 3 months or first year of operation) Ongoing (per year of operation) By level of severity By category or cause, e.g.: requirements defect, design defect, code defect, documentation/on-line help defect, defect introduced by fixes, etc.

3. Reliability Availability (percentage of time a system is available, versus the time the system is needed to

Page 20


be available) Mean time between failure (MTBF) Mean time to repair (MTTR) Reliability ratio (MTBF / MTTR) Number of product recalls or fix releases Number of production re-runs as a ratio of production runs

10. Write a brief note on “Benchmarking”. (Points expected: What is it? Who is it for? How to implement? And Benefit realization)

What is Benchmarking? Benchmarking is the process of comparing one's business processes and performance metrics to industry bests or best practices from other companies. It is a systematic process for identifying and implementing best or better practices. Although experts break benchmarking into several types, there exist two main types; “Informal" and "Formal" Benchmarking.

What is Informal Benchmarking? This is a type of benchmarking that most of us do unconsciously at work and in our home life. We constantly compare and learn from the behavior and practices of others – whether it is how to use a software program, how to cook a better meal, or play our favorite sport. In the context of work, most learning from informal benchmarking comes from the following:Talking to work colleagues and learning from their experience (coffee breaks and team meetings are a great place to network and learn from others). Consulting with experts (for example, business consultants who have experience of implementing a particular process or activity in many business environments.Networking with other people from other organizations at conferences, seminars, and Internet forums.On-line databases/web sites, such as the BPIR, and publications that share benchmarking information provide quick and easy ways to learn of best practices and benchmarks.

What is Formal Benchmarking?There are two types of Formal Benchmarking - Performance and Best Practice Benchmarking.Performance benchmarking: This involves comparing the performance levels of organizations for a specific process. This information can then be used for identifying opportunities for improvement and/or setting performance targets. Performance levels of other organizations are normally called benchmarks and the ideal benchmark is one that originates from an organization recognized as being a leader in the related area. Performance benchmarking may involve the comparison of financial measures (such as expenditure, cost of labor, cost of buildings/equipment, cost of energy, adherence to budget, cash flow, revenue collected) or non-financial measures (such as absenteeism, staff turnover, the percentage of administrative staff to front-line staff, budget processing time, complaints, environmental impact or call center performance).Best practice benchmarking: This is where organizations search for and study organizations that are high performers in particular areas of interest. The processes themselves of these organizations are studied rather than just the associated performance levels, normally through some mutually beneficial agreement that follows a benchmarking code of conduct. Knowledge gained through the study is taken back to the organization and where feasible and appropriate, these high performing or best practices are adapted and incorporated into the organization’s own processes. Therefore best practice benchmarking involves the whole process of identifying,

Page 21


capturing, analyzing, and implementing best practices. There are a number of best practices benchmarking methodologies. One of which is the TRADE Best Practice Benchmarking Methodology.

Who uses Benchmarking?In the West most large and highly successful organizations use best practice benchmarking as a tool to continually learn and improve. The resources needed to carry out repeated best practice benchmarking projects properly and in a way that maximizes the learning to be gained from the experiences can be considerable; hence it is used more frequently within large organizations. A key reason for the development of the BPIR was to offer help to all organizations (large or small) who may not have the necessary resources to undertake best practice benchmarking. Therefore the BPIR website has been designed to assist in every step of a benchmarking process. On the other hand, comparative or competitor benchmarking is not affected to the same degree by resources, and is used by organizations of all sizes, the most basic form of this practice is simply knowing your main competitors product price, something that is a prerequisite to staying in business.

Indications are that the use of benchmarking worldwide continues to grow since Robert Camp wrote the first book on benchmarking in 1989. Support for this comes from

The 2008 study by the Global Benchmarking Network showed the improvement tools that are likely to increase in popularity the most over the next three years are Performance Benchmarking, Informal Benchmarking, Strengths, Weaknesses, Opportunities, and Threats, and Best Practice Benchmarking. Current use of Informal benchmarking is 68% of organizations , Performance benchmarking, 49%, and Best practice benchmarking, 39%.The growth from year to year in membership of the Global Benchmarking Network which now has representatives from over 20 countriesThe growth in the number of countries that have a business excellence award to more than 70 (the growth in business excellence is likely to be correlated to the growth in benchmarking as a central part of business excellence is benchmarking with as much as 50% of the points associated with these models attributed to benchmarking) andThe continuing popularity of benchmarking within the academic community as the number of papers written on the subject continues to grow

A Benchmarking ProcessNow that we have the basic objectives and the definitions, we need a process to achieve the objectives; such a process provides the means for achieving the ends outlined by our objectives.What might such benchmarking process look like? While there are many alternatives, consider the following:

Page 22


Defining and Planning the ProjectYou need to define the project in precise terms and develop a complete, yet simple, project plan. Start with a preliminary plan and build it over time to the appropriate level of preciseness. Such a plan should include a way to measure your success. A project like benchmarking is like (and should probably be managed like) any other project you undertake. Be sure to include in your project plan items such as project objectives, scope, approach, timeline, and budget.Understanding Where You AreIn order to utilize information about how others are doing, you need to first understand how you are doing or, at least, how you would like to be doing. This requires that you have performance measures or Metrics (see How to Measure Success -- Uncovering the Secrets of Effective Metrics) so that you can judge how you are doing.Given these measures, you can use them to help organize your project and to select your benchmarking partners. You can use these measures to guide your search for secondary data, to help generate your preliminary questionnaire, and to conduct a preliminary survey to narrow the field in your search for potential partners.Understanding Where You Can BeBased your preliminary studies, you need to select potential partners, ascertain their willingness to participate, and develop your final questionnaire. The questions should help you focus on the specifics of what you want to learn.To get the most out of an exercise like this you have to have the "right" people participate, both from your team, as well as those of your partners. The right people means the best combination of technical and people skills so that you can both elicit and understand the information you are gathering.Once you have your team, you can proceed to schedule and conduct the information exchanges with the several partners you've identified.Two points to remember:

Page 23


Benchmarking is a search for how, as well as how much. To replicate results in your organization you need to understand how they have been achieved by others, andBenchmarking need not require you to visit others. You can achieve the results in many ways, depending on the time and resources available to you. The following chart outlines several alternatives for conducting exchanges. As more time and resources are available and as the need increases, you can elect to use the more sophisticated and time-consuming processes.

It is through these processes that you gather the data to determine where you can be.And the next question is, "How soon can I expect to see some results?" The following table gives some ideas of time frame, based on our experience. How Soon You Need Results Benchmarking Alternatives Within a week Reading library research

Surfing the web Telephone interviews

One to two weeks Research by a professional librarian Hire a consultant

Three to six weeks Rapid Benchmarking* Traditional site visit (2 or 3 sites only)

Two or more months Traditional benchmarking Identifying Lessons LearnedNow that you know how others are doing, you can use the data to understand how you can improve. The most straight-forward way is to assess where there are gaps between your performance and that of your benchmarking partners. Further, you can use these assessments to identify best practices, in particular ones you'd like your organization to adopt.Applying the Lessons LearnedYou are ready to begin implementing what you've learned. This is the "next step."

Page 24


This is where the rubber hits the road. You've learned what others are doing and how they are doing it. You need to ensure that all relevant staff in you reorganization is aware of and can make use of what you've learned. Your report and your presentations may in fact be one of the most important activities in your project.SummaryWe've defined benchmarking and provided an overview of a process that you can follow. The process allows you to understand where you are and where you can be, and then provides a view of how you can identify the lessons learned in your study. These are the best practices. They are what you can form the basis for improving your process for moving it to where it needs to be.

Benefits of Benchmarking

Benchmarking is a common practice and sensible exercise to establish baselines, define best practices, identify improvement opportunities and create a competitive environment within the organization. Benchmarking helps companies:

Lowering Labor CostsOne advantage of benchmarking may be lower labor costs. For example, a small manufacturing company may study how a top competitor uses robots for several basic plant functions. These robots may help the competitor save a significant amount of money on labor costs. Company managers may obtain information on these robotics systems through the competitor's website or online articles. They may also identify the company that sold the competitor the robots. Subsequently, the company using benchmarking may call the robot manufacturer to help set up its own system.

Improving Product QualityCompanies may also use benchmarking to improve product quality. Engineers sometimes purchase leading competitors' products. They may then take them apart, study them and determine how the competitors' products outlast or outperform others in the industry. Chemical engineers may study food or cleaning products in a similar manner. They can then compare various elements contained in competitive products to their own product line. Subsequently, improvements can be made to product quality.

Increasing Sales and ProfitsA company that uses benchmarking to improve its functions, operations, products and services may enjoy increases in sales and profits. Customers are likely to notice these improvements. The benchmarking company may also promote is improvements through company brochures, its sales reps, magazine and television ads. These efforts are likely to increase sales, especially among core customers. Companies that operate more efficiently due to benchmarking can drastically lower their expenses. These savings can be lead to greater profits.

ConsiderationsSome organizations use internal benchmarking to improve performance in different departments. Department managers may study and emulate the best practices of one particular department. These changes may spark improvements among all departments. Internal benchmarking has its limitations, however. The company's top

Page 25


department may not be functioning as efficiently as others in the industry. This means the other departments were not truly benchmarking against the best departments out there.

1. Understand your performance relative to close competitors

Having a thorough understanding of your own performance can only get you so far. For instance, if you’re working to improve year-over-year new product introduction defect rates, it might benefit you to understand the current industry average. Where one or two percentage points can have a dramatic impact, that intelligence may warrant an investment or reallocation of resources.

2. Compare performance between product lines/business units in your own company

Benchmarking doesn’t necessarily have to be an exercise that requires competitive intelligence. Many companies—especially large and distributed ones—benchmark performance of facilities and products having similar processes as well as metrics and KPIs. Again, this analysis can lead to deeper investigations as to why a particular facility, product, or business unit is underperforming.

3. Hold people more responsible for their performance

Without an internal or external benchmark for comparing performance, it can be a challenge to set precedents every year. Benchmarking projects and reports give you perspective on what’s considered “good” performance, and can be an instrumental tool for measuring the effectiveness of facilities, product lines, business units, and even particular personnel.

4. Drill down into performance gaps to identify areas for improvement

Even benchmarking a high-level metric such as overall equipment effectiveness (OEE) can result in some serious discussions amongst leadership. Many companies carry out such benchmarking projects, and then drill down into the variables to identify where the real culprits of underperformance reside. OEE, for example, can be broken down into components of quality, availability, and efficiency. A disparity between industry averages could surface as a disparity in quality management process and/or software capabilities.

5. Develop a standardized set of processes and metrics

The process of undertaking a benchmarking project can encourage organizations to invest resources in standardizing the calculation of metrics and KPIs. The challenge is that metrics such as OEE and the cost of quality can be calculated in numerous ways. Whether it’s adopting industry standards or just making sure calculations are standardized across your facilities, having a solid baseline for comparison is one of the keys to successful metrics program as well as benchmarking projects.

6. Enable a mindset and culture of continuous improvement

Providing metrics performance visibility to shop floor workers all the way up to the top floor allows personnel to understand how their actions impact certain areas of business. Adding an additional layer to those key

Page 26


performance indicators, showing them how their current performance compares to industry targets or even internal targets, can be incentive to drive productivity and innovation needed to exceed those averages.

7. Better understand what makes a company successful

Market leaders are the ones that exceed industry benchmarks. If you’re comparing on-time deliveries or first pass yield, benchmarking can provide a better outlook as to where you are versus where you want to be. The challenge is that successful companies are no doubt working to widen the gap.

Information System Audit and Control Practices

11. What do you understand by the term Information Systems Audit? You have been asked to conduct an IS audit for 3 locations of JB Technologies Ltd, a software development company, at multi-cities within India (17 locations), United Kingdom (3 locations), and the United States of America (13 locations).What is the basis on which you will choose your 3 locations? What areas (at least 2 main and 2 support functions) of organization’s business practices will need to be covered? List at least 3 things that you will cover in each of these business practices. Explain briefly justifying your stand taken above.

Information Systems AuditAn information technology audit, or information systems audit, is an examination of the management controls within an Information technology (IT) infrastructure. The evaluation of obtained evidence determines if the information systems are safeguarding assets, maintaining data integrity, and operating effectively to achieve the organization's goals or objectives. These reviews may be performed in conjunction with a financial statement audit, internal audit, or other form of attestation engagement.

IT audits are also known as "automated data processing (ADP) audits" and "computer audits". They were formerly called "electronic data processing (EDP) audits".

The primary functions of an IT audit are to evaluate the systems that are in place to guard an organization's information. Specifically, information technology audits are used to evaluate the organization's ability to protect its information assets and to properly dispense information to authorized parties. The IT audit aims to evaluate the following:Will the organization's computer systems be available for the business at all times when required? (known as availability) Will the information in the systems be disclosed only to authorized users? (known as security and confidentiality) Will the information provided by the system always be accurate, reliable, and timely? (measures the integrity) In this way, the audit hopes to assess the risk to the company's valuable asset (its information) and establish methods of minimizing those risks.

Choosing your 3 locations

Page 27


Sites will be chosen with the objective to maximize the audit scope on critical processes. The locations where the most number of processes can be conveniently audited are preferable.The choice of the sites is largely based with the objective to maximize the Audit scope and obtain samples from location where the maximum number of critical processes are operational. So for example we assume the US offices may be BD(Business Development) and sales and marketing locations, the Indian Offices may have the off-shore Development centers and the datacenters and DR locations. The UK offices could be other BD and minor development. It is essential to select processes which are most critical to the business to have the highest value derived from the audit. We also assume that all locations across US would have similar and standardized setups in IS infrastructure. Hence auditing one location would be similar to auditing all locations. Location in India would be chosen based on their criticality, the location with the Datacenter needs to be selected to audit the security and safeguarding controls implemented, and the remaining location can be a center where other operations originate

What areas (at least 2 main and 2 support functions) of organization’s business practices will need to be covered?

There would be 3 core processes of the organization1. Operations2. Management3. Support

Operations: Since JB is a Software development firm, the operations would include sub processes like (these are high-level, detailing is welcome)1. Research2. Design3. Development (Coding)4. Testing5. Release6. Support

Management Process: Some of the sub-process for Management includes:1. Corporate2. Finance3. Project-Management4. Audit5. Business Development6. Sales7. Communications (PR)

Support Process: Mostly cost-center processes1. IT2. Quality3. HR4. Admin (Facilities, Security, Transport, Vendor Management)

Page 28


Functions which can be chosen for audit can be1. Operations Processesa. Development functionsi. Segregation of development environments from the ii. Security measures in place for the code libraryiii. Has adequate training been provided for testing and Productions one developers

b. Testing Processesi. Use of standard practices and templates while creating test case, test scenarios, test scripts ii. Use of Responsibility Traceability matrix (RTM) to track the coverage of requirements and scenariosiii. Use of defect management tool to track the defects and implementation of defect prevention initiatives

3. Support Processesa. IT Team(Datacenter)i.Location of Datacenter and protection measures from ii. It team members who have access to the iii. Change management procedures employed by the

b. Admini. Environmental threats(Fire,flooding, heating, pests etc)ii. Datacenter and logical access to the systemsiii. Awareness of actions during emergency situations

c. IT teami.Vendor Contracts(SLA’s,)ii. Physical Securityiii. Evacuation procedures in times of disasters

We need to now expand each of the points mentioned above…

12. Describe briefly with suitable examples as to how introduction of Information SystemAudit can improve the organization’s:a. Safeguard of assets?b. System effectiveness?c. Data integrity?

13. Describe briefly the four major activities of Information System Audit Process.(Planning, Conducting of Audit, Reporting of Audit findings, and Follow-up)

An information technology audit, or information systems audit, is an examination of the management controls within an Information technology (IT) infrastructure. The evaluation of obtained evidence determines if the

Page 29


information systems are safeguarding assets, maintaining data integrity, and operating effectively to achieve the organization's goals or objectives. These reviews may be performed in conjunction with a financial statement audit, internal audit, or other form of attestation engagement.

PlanningDuring this phase, the audit team's goal is to develop a plan to guide their work on the audit. An audit plan covers:

the purpose of the audit (why it is being done); the scope of the audit (what they will and won't be examining); the criteria (how they will be evaluating the audit topic); and the timeline (how long each phase of the audit will take).

In the planning phase: The Auditor sends a notice of planning letter to the company. This is the official notice that an audit is

going to be conducted. The audit team identifies the expertise needed to develop the audit plan. In some cases, this may hire

subject matter experts to supplement the audit team’s knowledge and research. The audit team meets with the key contact and others as appropriate to discuss the audit topic and

confirm access to the information they will require for the planning phase (including staff interviews and document requests).

The audit team collects the required information and creates a draft audit plan, which is reviewed several times before being finalized.

The company informs the Auditor about the decision.

Conducting of Audit

Page 30


During this stage, all the participants must be ready to work together and have a good understanding of their roles and responsibilities. This will keep the process moving smoothly as the Examination phase requires frequent interactions between the audit team and staff at the audited organization.

During this phase: The Auditor sends a notice of examination letter to the organization which outlines the audit plan, confirms

the key contact and gives an overview of the audit team's professional responsibilities and auditing standards.

Each member of the audit team is assigned specific tasks and carries them out. These tasks include conducting interviews, requesting and reviewing documents, and other forms of data collection and analysis.

When the audit team has finished gathering evidence, they identify their key findings and conclusions. Once these have been developed, the audit team:

o sends them to the audited organization;o meets with the audited organization and documents this discussion;o reviews issues raised at the meeting and determines how to address them (which may include

further evidence gathering and making changes);o completes any final changes before moving to the Reporting phase.

Reporting of Audit findingsNow that the facts have been gathered and the key findings and conclusions are in place, the reporting phase begins. For reporting, different ways and various reporting methods may be discussed with your organization to identify the appropriate report for that particular audit. However, the traditional method is to produce a formal written report. This is done through the following process: The audit team produces a preliminary draft report, which is thoroughly reviewed by internal tem members. The audit team incorporates the feedback from the reviewers as required and submits the preliminary draft

reportfor review and approval to the Auditor General. The approved draft report is sent to the audited organization, which gets sometime to give feedback on the

report's accuracy and recommendations The audit team reviews feedback from the audited organization and determines what changes are required

in order to produce the final report. The final report is submitted to the Auditor General and the audited organization so that they can prepare

their respective comments for publication in the report.

Follow-upDuring this phase, Audit team will follow-up on the organization's progress in implementing the recommendations contained in the report.Follow-ups are a necessary process for ensuring that recommendations are addressed.Action Plans: Agencies are always asked to provide, within few months, an action plan describing how and when they will implement the recommendations. Self-Assessments:Audited organizations are sent a self-assessment form in which to describe the progress they have made in implementing the recommendations and their plans going forward. Organizations will

Page 31


have fewweeks to complete these short forms. However, subsequent follow-upsmay be required on outstanding recommendations or certain key recommendations that have not been satisfactorily addressed.

14. Explain the terms: (i) IT Service Management (ITSM) and (ii) IT Service Management System (ITSMS)? Describe briefly all 13 IT service processes of ISO 20000-1:2011 and map the same with software service activities.

IT service management (ITSM or IT services) refers to the implementation and management of quality IT services that meet the needs of the business. IT service management is performed by IT service providers through an appropriate mix of people, process and information technology. An IT Service Management System (ITSMS) is a systematic approach to managing the IT services delivered to customers (internal or external). It encompasses people, processes and IT systems.

ISO 20000 is the first international standard for Information Technology Service Management and is fully compatible and supportive of the ITIL (IT Infrastructure Library) framework. ISO/IEC 20000-1:2011 specifies four key service management processes broken into 13 IT processes:1. Service Delivery Processes – includes Service Level Management, Availability Management, and Capacity

Management, Service Reporting,Information Security Management , Budgeting and Accounting2. Relationship Processes – involves interfaces between the service provider and customers and suppliers3. Resolution Processes – focuses on incidents being resolved or prevented4. Control Processes – involves managing changes, assets, and configurations

Service Delivery Process:-a. Capacity Management:Adjustment of the capacity of a resource (equipment, machine, or system) to meet a

planned demand or load. In general, manufacturing capacity may be adjusted by working overtime or redeploying the manpower.In many organizations 24*7 support is required. So the capacity of the resource is adjusted to work in 3 shifts of 8 hours each so that the support is given for 24 hours.

b. Service Level Management:Implementing Service Level Management can only be completely successful when the other ITIL processes are implemented as well. The main aim of SLM is to ensure the quality of the IT services provided, at a cost acceptable to the business/customer.The goal for SLM is to maintain and improve on service quality through a constant cycle of agreeing, monitoring, reporting and improving the current levels of service. Software Example:- In software industry for various projects we have batch running for generating reports or processing large data for warehouses. Such batches have defined SLA which define the time within which the data or reports are expected.

c. Availability and Service Continuity Management:The two processes, availability and service continuity management, must ensure that the agreed objectives of availability and continuity for the customer can be met in every case. It is vital that all activities and expenditure, as well as theresources assigned for the implementation of the continuity and availability targets, should be coordinated with the requirements of the business. The availability must be recorded for monitoring in order to identify and document deviations from the defined targets. We also recommend that the effectiveness of improvement measures which have

Page 32


been introduced should then be reviewed. The availabilities and planned maintenance windows must be forecast in advance and communicated to all those involved. This will enable preventative maintenance to be carried out on a targeted basis.

We recommend that the service provider clearly defines at least the following points with each customer group:•Maximum accepted period without service•Maximum accepted period with reduced service• Accepted reduced service level during a defined recovery periodThe service continuity strategy must be reviewed jointly with the business representatives on a continual basis, at least however annually.

d. Service Reporting:A clear definition must be provided for all reports as to the intention and purpose of the report, its target groups and, in particular, the data sources. Reporting needs identified from customer requirements must be met.The success of all service management processes depends upon the utilization of the information from the service reports. The management decisions, together with corrective action, must be based on the results of the service reports and communicated to all relevant parties

e. Budgeting and Accounting:The aim of budgeting and accounting for IT services is to budget for and provide documentary evidence of the costs for service provision.The costs expended for the budgeting and accounting processes must be determined according to customer, service provider and supplier demand. The benefits of recording operational data must justify the expense.

f. Information Security Management:The objective of the information security management is to provide effective control and monitoring of the information security for all service activities. Information security is a system of guidelines and procedures for identifying, controlling and protecting information and all operating materials associated with its/their storage, transfer and processing.

Control Process:-a.Change Management:The aim of change management is to ensure that all changes are evaluated, approved, introduced and reviewed using stipulated methods. In this context the focus is on the efficient and prompt implementation with minimal risk to the operational business.The change management processes and procedures are intended to ensure that changes have a clearly defined and documented scope. Only changes which have an identified business benefit will be authorized. Changes should be planned on the basis of priority and potential risk. Changes to configurations must be verified during the implementation of the change.The status of the changes and planned dates for implementation form the basis for change and release planning. Information on dates should be communicated to the persons affected by the change.

b. Release Management:Whereas change management concentrates on controlling changes, release management prepares the planned changes for distribution. Release management should be integrated into the configuration and changemanagement processes in order to ensure that the releases and implemented changes are coordinated. Release management coordinates the activities of the service provider, suppliers and business

Page 33


cycles.The outcome of this is a plan for the supply of a release to the operational IT environment. The aim of the release management is to deliver, distribute and monitor one or more changes in a release to the operationalenvironment.One of the key tasks of release management process is to coordinate all the participating resources in order to hand over a release to a shared environment. In this context good planning and management is a basic prerequisite for packaging releases, their successful distribution as well as for having the associated impact and risks for the business and the IT under control.We recommend that all aspects of the release be planned in advance with the business. To this end the impact on the associated CIs must be evaluated and both the technical as well as the non-technical aspects be jointly taken into account.For the purpose of transparency all release elements must be traceable and safeguarded to prevent their being changed. Only tested and approved releases should be accepted within the operationalenvironment.

c.Configuration Management:The aim of configuration management is to define and control the components of the service and infrastructure and to manage precise configuration information.All key assets and configurations should be assigned to the responsibility of a manager who ensures appropriate security and control. This is intended to guarantee, amongst other things, that approval is obtained before changes to the CI are implemented. The following recommendations for meeting the specifications for the configuration management process have become established practice:1. Planning and implementation2. Configuration identification3. Configuration control4. Proof of status5. Verification and audit

Relationship Process:-The relationship processes describe the two aspects of business relationship management and supplier management. In this context the standard focuses on the role of the service provider (frequently a company’s IT organization) which is logically positioned between customer and supplier.Both customers as well as suppliers can be part of the service provider’s organization or external. A fundamental distinction is drawn between the following three levels for the contracts:• The agreements between the customer and service provider are known as service level agreements (SLA).• External support (suppliers) required for the agreed IT services are formalized with underpinning contracts.• Operational level agreements govern the relationships within the IT organization for the service delivery.

In order to create good relationships between the participating parties’ clear agreements must be in place. In this context, all parties should have the same understanding of the business requirements, service capacity as well as the framework conditions and the respective responsibilities and obligations. This is the only way in which each party can meet its performance obligation.

a. Supplier Management:The aim of supplier management is to control suppliers in order to ensure a smooth delivery of high quality services.As a general rule there are a number of suppliers involved. These are often also

Page 34


subdivided into main suppliers and subcontract suppliers. It is therefore necessary to clearly define whether the service provider is to negotiate directly with all suppliers or whether a main supplier is to take over the responsibility for the subcontract suppliers.The supplier management process must ensure that the supplier understands its obligations to the service provider. The requirements must therefore be clearly defined and agreed. It is also necessary to ensure that all changes to these agreements are monitored by the change management process.In order to avoid conflicts we recommend that records be created of all official business transactions between all the parties. The services of the supplier must be continually monitored and an appropriate response taken as required.

b. Business Relationship Management:The aim of business relationship management is to understand the customer and the business process drivers and based on this to establish and maintain a good relationship between the service provider and the customer.Three key aspects must be anchored within the organization in order to meet the requirements demanded of business relationship management:• Regular service reviews• Service complaints procedure• Measurement of customer satisfactionThere is no separate business relationship management process in ITIL V3.

Resolution Process:-The resolution processes include the incident and problem management processes. These are standalone processes even if they are closely interlinked. Incident management deals with the restoration of the service for the service user. Problem management by contrast deals with the identification and elimination of root causes in the case of major or repeat disruptions and therefore ensures a permanent and stable service infrastructure

a. Incidence Management:The aim of incident management is to restore the agreed service for the business and respond to service enquiries as quickly as possible.In order to fulfill the specification requirements it is necessary to ensure that the incident management is designed as a reactive and proactive process that responds to error messages. The process must focus on the restoration of the IT service concerned and consciously not deal with the identification of the root cause.The incident process (incidents and service requests) comprises receiving calls, recording, prioritization, taking account of security provisions as well as following up on the incident processing status.It should also govern the agreement on fault processing with the customer as well as any escalation procedures. All incidents must be recorded in such a way as to enable the relevant information to rectify the error to be ascertained and analyzed.The progress of work should be reported to the current and any potential personnel affected. All activities must be fully recorded in the incident ticket.Wherever possible, customers must be able to continue their business in the appropriate way. Workarounds can also be utilized for this purpose

b. Problem Management: The aim of problem management is to minimize the disruption to and impact on the business by proactively identifying and analyzing the root causes of service incidents and by managing problems until these are rectified.

Page 35


Problem management must identify the root causes of the incidents on a reactive basis and proactively prevent incidents reoccurring. Problems are to be classified as known errors as soon as the root cause of the incident is known and a solution method for avoiding such incidents has been found.

For incident management to receive an optimum supply of information, all known errors and IT services affected must be documented and the associated configuration items identified. Known errors should only be closed once a definitive, successful solution has been found.

Once the root cause has been identified and a decision reached on the solution, this solution must be dealt with by the change management process. Information on the progress, potential workarounds or permanent solutions must be sent to all parties involved.

The closure of problem tickets should always be carried out in accordance with the following reviews:• Has the solution been precisely documented?• Has the root cause been categorized in order to provide support for future further analyses?• Have the customers and support employees affected been informed of the solution?• Has the customer confirmed that he/she accepts the solution?• Has the customer been informed if no solution has been found?The effectiveness of completed solutions to problems must be reviewed. In particular, trends such as for example reoccurring problems and incidents, defects, errors, known errors in planned releases or resource commitments must be identified by employees.

15. How does control of “Change” and “Configuration” help in controlling Quality ofServices in Business Application Releases?

16. Explain the terms: (i) Information Security (IS), (ii) Information Security Management (ISM) and Information Security Management System (ISMS)? Describe briefly the PDCA cycle of ISO 27001:2005 and its 11 control areas or domains.

Information SecurityInformation security, sometimes shortened to InfoSec, is the practice of defending information from unauthorized access, use, disclosure, disruption, modification, perusal, inspection, recording or destruction. It is a general term that can be used regardless of the form the data may take (electronic, physical, etc.)“It is preservation of confidentiality, integrity and availability of information. In addition, other properties, such as authenticity, accountability, non-repudiation and reliability can also be involved." (ISO/IEC 27000:2009)Information is an asset to all individuals and businesses. Information Security refers to the protection of these assets in order to achieve C - I - A:Confidentiality - protecting information from being disclosed to unauthorized parties.Integrity - protecting information from being changed by unauthorized parties.Availability - to the availability of information to authorized parties only when requested.

Page 36


Information Security Management (ISM) describes controls that an organization needs to implement to ensure that it is sensibly managing these risks.The risks to these assets can be calculated by analysis of the following issues: Threats to your assets. These are unwanted events that could cause the deliberate or accidental loss,

damage or misuse of the assets Vulnerabilities. How susceptible your assets are to attack Impact. The magnitude of the potential loss or the seriousness of the event.Standards that are available to assist organizations implement the appropriate programs and controls to mitigate these risks are for example BS7799/ISO 17799, Information Technology Infrastructure Library and COBIT.Information Security Management System (ISMS)

An information security management system (ISMS) is a set of policies and procedures for systematically managing an organization's sensitive data. The goal of an ISMS is to minimize risk and ensure business continuity by pro-actively limiting the impact of a security breach. It is a systematic approach to managing sensitive company information so that it remains secure. It includes people, processes and IT systems by applying a risk management process. It can help small, medium and large businesses in any sector keep information assets secure. Using this family of standards will help your organization manage the security of assets such as financial information, intellectual property, employee details or information entrusted to you by third parties.

‘Information is an ASSET which, like other important business assets, has VALUE to an organization and consequently needs to be SUITABLY protected’ “Information Security Management System is that part of the overall management system, based on a business risk approach, to establish, implement, operate, monitor, review, maintain and improve information security’ ISMS always follows Plan-Do-Check-Act methodology (PDCA). The Plan phase is about designing the ISMS, assessing information security risks and selecting appropriate

controls. The Do phase involves implementing and operating the controls. The Check phase objective is to review and evaluate the performance (efficiency and effectiveness) of the

ISMS. In the Act phase, changes are made where necessary to bring the ISMS back to peak performance

Page 37


Implementation PhasesVarious IT initiatives that can save time and cost on implementation phases are illustrated in figure 2. As explained earlier, an organization also needs to have the detailed understanding of PDCA implementation phases to manage the costs of the project. The cycle of PDCA is consistent with all auditable international standards: ISO 18001, 9001 and 14001. ISO/IEC 27001:2005 dictates the following PDCA steps for an organization to follow:• Define an ISMS policy.• Define the scope of the ISMS.• Perform a security risk assessment.• Manage the identified risk.• Select controls to be implemented and applied.• Prepare an SOA.These suggested PDCA steps are further simplified and mapped (figures 1, 3 and 4) to the implementation phases developed for easy understanding and implementation—with the end objective of time and cost savings in mind. The following steps take into account the IT maturity within the organization and the review/registration process (see figure 4 for the details of review and registration steps).

Phase 1 - Identify Business Objectives: Stakeholders must buy in; identifying and prioritizing objectives is the step that will gain management support. Primary objectives can be derived from the company’s mission, strategic plan and IT goals. The objectives can be:• Increased marketing potential• Assurance to the business partners of the organization’s status with respect to information security

Phase 2 - Obtain Management Support: Management must make a commitment to the establishment, planning, implementation, operation, monitoring, review, maintenance and improvement of the ISMS. Commitment must

Page 38


include activities such as ensuring that the proper resources are available to work on the ISMS and that all employees affected by the ISMS have the proper training, awareness and competency.

Phase 3 - Select the Proper Scope of Implementation: Only the processes, business units, and external vendors or contractors falling within the scope of implementation must be specified for certification to occur. The standard also requires companies to list any scope exclusions and the reasons why they were excluded. Identifying the scope of implementation can save the organization time and money.

Phase 4 - Define a Method of Risk Assessment: To meet the requirements, companies need to define and document a method of risk assessment. The ISO/IEC 27001 standard does not specify the risk assessment method to be used. ISO 27001 needs risk evaluations based on levels of confidentiality, integrity and availability (CIA):• Confidentiality: Ensuring that information is accessible only to those authorized to have access• Integrity: Safeguarding the accuracy and completeness of information and processing methods• Availability: Ensuring that authorized users have access to information and associated assets when required

Phase 5 - Prepare an Inventory of Information Assets to Protect, and Rank Assets According to Risk Classification Based on Risk Assessment: The Company needs to create a list of information assets to be protected. The risk associated with assets, along with the owners, location, criticality and replacement value of assets, should be identified. Information regarding the grouping of assets, data classification documents and assets inventory documents will be useful.

Phase 6 - Manage the Risks, and Create a Risk Treatment Plan: To control the impact associated with risk, the organization must accept, avoid, transfer or reduce the risk to an acceptable level using risk mitigating controls. The next stage is performing the gap analysis. It is important to obtain management approval of the proposed residual risks.

Phase 7 - Set Up Policies and Procedures to Control Risks: For the controls adopted, as shown in the SOA, the organization will need statements of policy or a detailed procedure and responsibility document to identify user roles for consistent and effective implementation of policies and procedures. Documentation of policies and procedures is a requirement of ISO/IEC 27001. The list of applicable policies and procedures depends on the organization’s structure, locations and assets.

Phase 8 - Allocate Resources, and Train the Staff: The ISMS process highlights one of the important commitments for management: sufficient resources to manage, develop, maintain and implement the ISMS. It is essential to document the training for audit

Phase 9 - Monitor the Implementation of the ISMS: The periodic internal audit is a must for monitoring and review. Internal audit review consists of testing of controls and identifying corrective/preventive actions. To complete the PDCA cycle, the gaps identified in the internal audit must be addressed by identifying the corrective and preventive controls needed and the company’s compliance based on a gap analysis.

Phase 10 - Prepare for the Certification Audit: In order for the organization to be certified, it is essential that it conduct a full cycle of internal audits, management reviews and activities in the PDCA process, and that it

Page 39


retains evidence of the responses taken as a result of those reviews and audits. ISMS management should review risk assessments, the RTP, the SOA, and policies and procedures at least annually.

Phase 11 - Conduct Periodic Reassessment Audits: Follow-up reviews or periodic audits confirm that the organization remains in compliance with the standard. Certification maintenance requires periodic reassessment audits to confirm that the ISMS continues to operate as specified and intended. As with any other ISO standard, ISO 27001 follows the PDCA cycle and assists ISMS management in knowing how far and how well the enterprise has progressed along this cycle. This directly influences the time and cost estimates related to achieving compliance.

17. Explain the terms: (i) Business Continuity and (ii) Business Continuity Management System (BCMS)? Describe the five elements (stages) of a BCM program management and map them for an Information management or software development company.

Business Continuity (BC):The capability of the organization to continuedelivery of products or services at acceptablepredefined levels following a disruptiveincidentBC is about building and improvingresilience in your business; it’s about identifying your key products and services and the mosturgent activities that underpin them and then, once that ‘analysis’ is complete, it is about devisingplans and strategies that will enable you to continue your business operations and enable you to recoverquickly and effectively from any type disruption whatever its size or cause. It gives you a solid framework to lean on in times of crisis and provides stability and security. In fact, embedding BC into your business is proven to bring business benefits.

Business Continuity Management System (BCMS):Business Continuity Management (BCM) is defined as a holistic management process that identifies potential threats to an organization and the impacts to business operations those threats, if realized, might cause, and which provides a framework for building organizational resilience with the capability of an effectiveresponse that safeguards the interests of its key stakeholders, reputation, brand and value-creating activities.

5 Elements or Stages of a BCMPhase 1: Identify the risksThe first phase is to conduct a risk assessment, identifying any potential hazards that could disrupt your business. Consider any type of risk your team can imagine, including natural threats, human threats and technical threats.Phase 2: Analyze the risks you faceNext, you’ll perform a business impact analysis (BIA) to gauge the impact of each potential risk. For each risk, determine how severe the impact would be and how long your business could survive without those processes running. Consider what is absolutely necessary for recovery, how quickly it needs to happen, what are your minimum operating resources are and any dependencies, either internal or external.Phase 3: Design your strategyNow it’s time to figure out strategies to mitigate interruptions and to quickly recover from them. Consider everything you’ll need to protect your people, your assets and you’re your functions. Start by comparing your current recovery capabilities to your business requirements and plan how you will fill that gap.

Page 40


Phase 4: Plan development and executionFinally, it’s time to create a concise, well organized and easy-to follow document or set of documents. Consider everyone that may use the plan, and document it in a way that will be most useful when your business is suffering an interruption. Then publish the plan, socialize it and train your staff on how to use itPhase 5: Measure your success by testingA plan isn’t truly a plan until it has been thoroughly tested. There are a variety of tests you should perform, with each providing different information on how to improve your plan. Tests can range from a checklist test, a walk-through performed by you your team as if there were an actual event, emergency evacuation drills, and when ready, a full on recovery simulation test is a bit more complex and involves your team simulating and emergency and using the actual equipment, facilities and supplies just as in a real disaster. After each test, you can make any necessary modifications to your plan to keep it current.

18. Explain the terms: (i) Control and (ii) Control Objectives. Expand the word COBIT? Name the four major areas of COBIT and explain your understanding on each one of them.

ControlA control is any action taken by management, the board, and other parties to manage risk and increase the likelihood that established objectives and goals will be achieved.. Controls can be preventative, detective, or corrective and can be fully automated, procedural, or technology-assisted human-initiated activities. They can include actions, devices, procedures, techniques, or other measures. Also used as a synonym for safeguard or countermeasure

Control ObjectiveA Control Objective is a statement of the desired result or purpose,which is to be achieved by implementing the control procedures in a particular process. It is the aim or purpose of specified controls at the service organization which address the very risks that these controls are intended to effectively mitigate

COBIT Control Objectives for Information and Related Technology (COBIT) is a framework created by ISACA for information technology (IT) management and IT governance. It is a supporting toolset that allows managers to bridge the gap between control requirements, technical issues and business risks.

In recent years, it has become increasingly evident that there is a need for a reference framework for developing and managing internal controls and appropriate levels of security in information technology (IT). The application of IT has become central to the strategy and business processes of many entities. As such, successful organizations require an appreciation for and a basic understanding of the risks and constraints of IT at all levels within the enterprise in order to achieve effective direction and adequate controls. COBIT (Control Objectives for Information and related Technology) provides such a control and security framework for IT.

Page 41


The framework provides good practices across a domain and process framework.The business orientation of COBIT consists of linking business goals to IT goals, providing metrics and maturity models to measure their achievement, and identifying the associated responsibilities of business and IT process owners. Cobit subdivides IT into four areas

1. Plan and organize—This domain covers strategy and tactics, and concerns the identification of the way IT can best contribute to the achievement of the business objectives. Furthermore, the realization of the strategic vision needs to be planned, communicated and managed for different perspectives. Finally, a proper organization as well as technological infrastructure must be put in place.

2. Acquire and implement—To realize the IT strategy, IT solutions need to be identified, developed or acquired, as well as implemented and integrated into the business process. In addition, changes in and maintenance of existing systems are covered by this domain to make sure that the life cycle is continued for these systems.

3. Deliver and support—This domain is concerned with the actual delivery of required services, which range from traditional operations over security and continuity aspects to training. In order to deliver services, the necessary support processes must be set up. This domain includes the actual processing of data by application systems, often classified under application controls.

4. Monitor and evaluate—All IT processes need to be regularly assessed over time for their quality and compliance with control requirements. This domain thus addresses management’s oversight of the organization’s control process and independent assurance provided by internal and external audit or obtained from alternative sources.

Page 42


34 processes in line with the responsibility areas of plan, build, run and monitor. The IT processes identified in COBIT can be applied at different levels within an organization. For example, some of these processes will be applied at the enterprise level, others at the IT function level and still others at the business process owner level.

19. Name and explain in brief (one or two sentences) the five principles of COBIT 5. Name and explain the seven enterprise enablers used in COBIT 5.

Principle 1: Meeting Stakeholder Needs: Enterprises have many stakeholders, and ‘creating value’ means different—and sometimes conflicting—

things to each of them. Governance is about negotiating and deciding amongst different stakeholders’ value interests. The governance system should consider all stakeholders when making benefit, resource and risk

assessment decisions. For each decision, the following can and should be asked:

o Who receives the benefits? o Who bears the risk? o What resources are required?

The COBIT 5 goals cascade translates stakeholder needs into specific, practical and customised goals within the context of the enterprise, IT-related goals and enabler goals.

Principle 2: Covering the Enterprise End-to-end: COBIT 5 addresses the governance and management of information and related technology from an

enterprisewide, end-to-end perspective. This means that COBIT 5:

Integrates governance of enterprise IT into enterprise governance, i.e., the governance system for enterprise IT proposed by COBIT 5 integrates seamlessly in any governance system because COBIT 5 aligns with the latest views on governance.

Covers all functions and processes within the enterprise; COBIT 5 does not focus only on the ‘IT function’, but treats information and related technologies as assets that need to be dealt with just like any other asset by everyone in the enterprise.

Principle 3: Applying a Single Integrated Framework: COBIT 5 aligns with the latest relevant other standards and frameworks used by enterprises:

Enterprise: COSO, COSO ERM, ISO/IEC 9000, ISO/IEC 31000 IT-related: ISO/IEC 38500, ITIL, ISO/IEC 27000 series, TOGAF, PMBOK/PRINCE2, CMMI

This allows the enterprise to use COBIT 5 as the overarching governance and management framework integrator.

Principle 4: Enabling a Holistic ApproachCOBIT 5 enablers are:

Factors that, individually and collectively, influence whether something will work—in the case of COBIT, governance and management over enterprise IT

Driven by the goals cascade, i.e., higher-level IT-related goals define what the different enablers should achieve

Systemic governance and management through interconnected enablers—To achieve the main objectives of the enterprise, it must always consider an interconnected set of enablers, i.e., each enabler:

Page 43


o Needs the input of other enablers to be fully effective, e.g., processes need information, organisational structures need skills and behaviour

o Delivers output to the benefit of other enablers, e.g., processes deliver information, skills and behaviour make processes efficient

Principle 5: Separating Governance from Management: The COBIT 5 framework makes a clear distinction between governance and management. These two disciplines:

Encompass different types of activities Require different organisational structures Serve different purposes

Governance—In most enterprises, governance is the responsibility of the board of directors under the leadership of the chairperson.

Management—In most enterprises, management is the responsibility of the executive management under the leadership of the CEO.

Governance ensures that stakeholders needs, conditions and options are evaluated to determine balanced, agreed-on enterprise objectives to be achieved; setting direction through prioritisation and decision making; and monitoring performance and compliance against agreed-on direction and objectives (EDM).

Management plans, builds, runs and monitors activities in alignment with the direction set by the governance body to achieve the enterprise objectives (PBRM).

7 enterprise enablers in Cobit 5 Processes—Describe an organised set of practices and activities to achieve certain objectives and

produce a set of outputs in support of achieving overall IT-related goals Organisational structures—Are the key decision-making entities in an organisation Culture, ethics and behaviour—Of individuals and of the organisation; very often underestimated as a

success factor in governance and management activities Principles, policies and frameworks—Are the vehicles to translate the desired behaviour into practical

guidance for day-to-day management Information—Is pervasive throughout any organisation, i.e., deals with all information produced and

used by the enterprise. Information is required for keeping the organisation running and well governed, but at the operational level, information is very often the key product of the enterprise itself.

Services, infrastructure and applications—Include the infrastructure, technology and applications that provide the enterprise with information technology processing and services

People, skills and competencies—Are linked to people and are required for successful completion of all activities and for making correct decisions and taking corrective actions

20. What is your understanding about (i) risk and (ii) risk management with respect to an IT enterprise? Explain how the terms threat, impact, etc. relate to risk. Explain briefly the activities that are carried out during risk identification, risk estimation, risk evaluation and risk treatment in the overall risk management process.

Risk is a probability or threat of damage, injury, liability, loss, or any other negative occurrence that is caused by external or internal vulnerabilities

Page 44


Risk management is the application of the principles of management to IT Risks in order to manage the risks associated with the field. IT risk management aims to manage the risks that come with the ownership, involvement, operation, influence, adoption and use of IT as part of a larger enterprise. IT risk management is a process done by IT managers to allow them to balance economic and operational costs related to using protective measures to achieve nominal gains in capability brought about by protecting the data and information systems that support an organization’s operations.

IdentificationRisks are about events that, when triggered, cause problems. Hence, risk identification can start with the source of our problems and those of our competitors (benefit), or with the problem itself.

Source analysis- Risk sources may be internal or external to the system that is the target of risk management (use mitigation instead of management since by its own definition risk deals with factors of decision-making that cannot be managed).

Examples of risk sources are: stakeholders of a project, employees of a company or the weather over an airport. Problem analysis - Risks are related to identified threats. For example: the threat of losing money, the

threat of abuse of confidential information or the threat of human errors, accidents and casualties. The threats may exist with various entities, most important with shareholders, customers and legislative bodies such as the government.

When either source or problem is known, the events that a source may trigger or the events that can lead to a problem can be investigated. For example: stakeholders withdrawing during a project may endanger funding of the project; confidential information may be stolen by employees even within a closed network; lightning striking an aircraft during takeoff may make all people on board immediate casualties.The chosen method of identifying risks may depend on culture, industry practice and compliance. The identification methods are formed by templates or the development of templates for identifying source, problem or event.

Common risk identification methods are:Risk Assessment

Objectives-based risk identification - Organizations and project teams have objectives. Any event that may endanger achieving an objective partly or completely is identified as risk.

Scenario-based risk identification - In scenario analysis different scenarios are created. The scenarios may be the alternative ways to achieve an objective, or an analysis of the interaction of forces in, for example, a market or battle. Any event that triggers an undesired scenario alternative is identified as risk – see Futures Studies for methodology used by Futurists.

Taxonomy-based risk identification - The taxonomy in taxonomy-based risk identification is a breakdown of possible risk sources. Based on the taxonomy and knowledge of best practices, a questionnaire is compiled. The answers to the questions reveal risks.

Common-risk checking - In several industries, lists with known risks are available. Each risk in the list can be checked for application to a particular situation.

Risk charting - This method combines the above approaches by listing resources at risk, threats to those resources, modifying factors which may increase or decrease the risk and consequences it is wished to avoid. Creating a matrix under these headings enables a variety of approaches. One can begin with

Page 45


resources and consider the threats they are exposed to and the consequences of each. Alternatively one can start with the threats and examine which resources they would affect, or one can begin with the consequences and determine which combination of threats and resources would be involved to bring them about.

Risk Assessment Once risks have been identified, they must then be assessed as to their potential severity of impact

(generally a negative impact, such as damage or loss) and to the probability of occurrence. These quantities can be either simple to measure, in the case of the value of a lost building, or impossible to know for sure in the case of the probability of an unlikely event occurring. Therefore, in the assessment process it is critical to make the best educated decisions in order to properly prioritize the implementation of the risk management plan.

The fundamental difficulty in risk assessment is determining the rate of occurrence since statistical information is not available on all kinds of past incidents. Furthermore, evaluating the severity of the consequences (impact) is often quite difficult for intangible assets. Asset valuation is another question that needs to be addressed. Thus, best educated opinions and available statistics are the primary sources of information. Nevertheless, risk assessment should produce such information for the management of the organization that the primary risks are easy to understand and that the risk management decisions may be prioritized. Thus, there have been several theories and attempts to quantify risks. Numerous different risk formulae exist, but perhaps the most widely accepted formula for risk quantification is:

Rate (or probability) of occurrence multiplied by the impact of the event equals risk magnitude

Risk TreatmentAccording to its definition, Risk Treatment is the process of selecting and implementing of measures to modify risk. Risk treatment measures can include avoiding, optimizing, transferring or retaining risk. The measures (i.e. security measurements) can be selected out of sets of security measurements that are used within the The options available for the treatment of risks include:

Retain/accept the risk - if, after controls are put in place, the remaining risk is deemed acceptable to the organization, the risk can be retained. However, plans should be put in place to manage/fund the consequences of the risk should it occur.

Reduce the Likelihood of the risk occurring - by preventative maintenance, audit & compliance programs, supervision, contract conditions, policies & procedures, testing, investment & portfolio management, training of staff, technical controls and quality assurance programs etc.

Reduce the Consequences of the risk occurring - through contingency planning, contract conditions, disaster recovery & business continuity plans, off-site back-up, public relations, emergency procedures and staff training etc.

Transfer the risk - this involves another party bearing or sharing some part of the risk by the use of contracts, insurance, outsourcing, joint ventures or partnerships etc.

Avoid the risk - decide not to proceed with the activity likely to generate the risk, where this is practicable.

Page 46


21. What constitute an eCommerce activity(ies)? What will you audit in an eCommerce environment? Describe the content of an audit report with respect to the objective of the audit, outcome or audit findings for an eCommerce business.

Electronic commerce, commonly known as e-commerce or eCommerce, is a type of industry where the buying and selling of products or services is conducted over electronic systems such as the Internet and other computer networks. Modern electronic commerce typically uses the World Wide Web at least at one point in the transaction's life-cycle, although it may encompass a wider range of technologies such as e-mail, mobile devices social media, and telephones as well.Components of an E-commerce environment:

o Networko Datao Databaseo E-commerce softwareo Server softwareo Server Operating systemo Server hardware

While auditing an ecommerce environment the focus will be on:

Page 47


a. Security: The system needs to be protected from unauthorized access, both logically and physically. With e-commerce specifically, information is to be made available only to those who need the access to complete the transaction or services, or follow up on questions and issues that may arise.

b. Availability. The system is available for operation and use as committed or agreed. This in itself does not set a minimum acceptable performance level for system availability - that is established through commitments made by mutual agreements between the related parties within the e-commerce business.

c. Processing integrity: The system processing is complete, accurate, timely, and authorized. It should be performing its intended function in an unimpaired manner that is free from unauthorized or inadvertent manipulation.

d.Confidentiality. Information that is being communicated and exchanged is protected as committed or agreed by the partners.

e. Privacy. Personal information collected from the client’s customers, employees, and other individuals is used, retained, and disclosed in conformity with the commitments in the entity’s privacy notice and with criteria set forth in Generally Accepted Privacy Principles issued by the AICPA/CICA

Content of an audit report:The auditor's report is a formal opinion, or disclaimer thereof, issued by either an internal auditor or an independent external auditor as a result of an internal or external audit or evaluation performed

The primary purpose of the audit for an ecommerce environment would be to assess the effectiveness and efficiency of security measures and their compliance with any of the accepted security standards and Operational Standards. The objectives follow the audit of Security and Audit Guide to Information Technology Security and include the assurances that: •A management control framework exists; •An effective security program is in place; •Security education and training is adequate; •Information/communications is appropriately classified and protected; •An effective personnel screening program is enforced; •Security breaches are dealt with; •Physical safeguards are in place for the protection of personnel and assets; •Contingency management has been developed; •Security requirements are met in contract management; and •Threat and risk assessments are conducted on a regular basis and prior to major system, application and telecommunication changes.

The Audit Findings are based on written documentation, and are sometimes more difficult to prioritize, hence, there is a need to prioritize the findings into categories. Throughout industry, audit findings are generally placed into one of four categories.

The first category is “Major” or “serious” findings. These are findings that if not addressed will lead or will very possibly lead to critical negative business impact. These are the first priority for an Ecommerce establishment to address. An example of a serious finding is that when customer data has not been secured and have high risk of data leakage or loss. This can invite legal fines and penalties as well which highly damage the brand and reputation of the business

Page 48


The second category for findings is “somewhat serious or somewhat major”. These types of findings can be serious if something out of the ordinary or of a non-routine nature happens. Frequently these types of findings are based on “what if” scenarios. What are the measures taken to prepare for a downtime, for unexpecected changes, or even a DDos attack on the site.Often these types of situations and “what if” scenarios are addressed by redundant measures in a process or on a piece of equipment, but these secondary safety features must frequently be in place, as human error, equipment failures do occur.

The third category is “Minor” findings. These are frequently small items that are easily overlooked. Examples include items where the documentation does not exactly match what the is operational. During an audit, this difference would be noted. Frequently, administrations have already come up with solutions to address a concern, but the documentation was never changed or updated. A minor finding might be that not every employee was provided their annual refresher training for security

The fourth category is “Scope for improvement” findings. These are simple errors, usually in documentation (spelling, grammar, references that do not take you directly to where you need to be, or incorrect dates). Awareness findings could also be errors where the auditors or inspectors feel that management should be aware of something,

Page 49

Documents

Quality Assurance - Web viewExpand the word COBIT? ... Such a system formed the basis for lessons for top management in Japan in 1950 and in subsequent ... Slogans