INTELLIGENT SOFTWARE MEASUREMENT SYSTEM (ISMS)

Junling Huang, Behrouz Homayoun Far Department of Electrical and Computer Engineering

University of Calgary, Alberta, Canada E-mail: {huaj, far}@ucalgary.ca

Abstract

Intelligent Software Measurement System (ISMS) is a multi-agent knowledge-based system that automates the ten-step Goal-Driven process to produce a software measurement plan based on user’s particular business goal(s). The initial task of converting a high-level business goal to several low-level measurement goals has already been accomplished. In the second phase of the ISMS project we concentrate on the core task of automating the conversion of a set of measurement goals to a software measurement plan which involves measures and corresponding actions. The ISMS architecture and software agents (PAs and EAs) are further designed to facilitate the process of the second phase of ISMS with respect to system learning and knowledge updating. Considering the variety and complexity of software measures and actions, an ontology for software measurement is defined and used to build the ISMS software measurement knowledge base.

Keywords: Multi-agent system (MAS); ISMS ontology; Goal-Driven software measurement process.

1. Introduction

A practical goal-oriented software measurement program plays a critical role in the success of managing software development process and products. However, building such a program, particularly making a realistic software measurement plan is a knowledge-intensive and resource-consuming task. It needs an experienced software measurement team to go through several iterations and feedback sessions to elicit software measurement knowledge and devise a practical measurement plan [1].

Intelligent Software Measurement system (ISMS) is a research project designed to automatically produce the software measurement plan towards user’s business goal(s) [1] [2]. It is divided into three phases. The first two phases fulfill the ten-step Goal-Driven software measurement process [3] together with building the software measurement knowledge base infrastructure. The third phase is the future extension of the system’s learning and knowledge base updating.

The main task of the initial phase is to convert the high-level business goal into several measurement goals with specific

software entities, attributes, purposes, perspectives and environment descriptions [1]. It covers from step one to step five of the Goal-Driven process, and has already been accomplished [2]. The knowledge base building for this phase has been presented in [1][2].

The second phase of ISMS is now well under way. It concentrates on the task of converting the measurement goal into a software measurement plan. It starts with identifying quantifiable questions and related indicators with respect to particular measurement goals (step 6), continues with identifying and defining required software data elements or measures to help answer those questions (step 7 & 8), and ends with identifying implementing actions and generating the software measurement plan (step 9 & 10) [3]. The main software measurement knowledge involved in this phase is a variety of software measures and corresponding actions. The acquisition of the knowledge is mainly based on literature review and authors’ experience. Due to the volume and complexity of the knowledge and future extendibility concerns we decided to organize and represent the knowledge using software measurement ontology. The ISMS ontology defined in this phase is a task ontology aimed at facilitating the last five steps of the Goal-Driven process, and it would be described further in Section 3 of this paper.

The structure of this paper is as follows. In section 2, in order to give a general idea about how ISMS works the system architecture is described. In Section 3, the building of ISMS ontology is presented to show how the knowledge is organized. The conclusions and future work is presented in Section 4.

2. System Architecture

ISMS is a multi-agent system (MAS) with two sets of agents — the user assistant agents and the measurement expert agents [2][4]. Considering the current needs and the third phase of ISMS, we design the system architecture further based on the initial design [2][4]. The current system architecture is shown in Fig. 1.

The current system has four types of users. The public user, expert user, and administrator have the same functionality as described in [2]. The main function of the knowledge analyst in the current system is to perform knowledge analysis and updating based on the data of system knowledge usage so that the quality of the knowledge base can be gradually improved.

0-7803-8886-0/05/$20.00 ©2005 IEEECCECE/CCGEI, Saskatoon, May 2005

1033

There are four groups of agents in the current ISMS. On the user side, the personal assistant agent (PA) group includes user PA and analyst PA. User PA mainly facilitates Goal-driven process and the communication between users and server side agents [2]. Analyst PA has all the abilities of user PA, plus extra ability of communicating with knowledge analysis agent to get and generate displays of the analysis data of system knowledge usage for knowledge analyst.

Fig. 1. System architecture

On the server side, there are three groups of expert agents (EAs), including facilitator group, Goal-Driven (GD) process group, and analysis group.

• The facilitator group includes directory agent and message exchange agent, which have abilities to manage agent addresses, services, and communications [2].

• The Goal-Driven (GD) process group has goal agent, measure agent and plan agent. Goal agent derives measurement goals from business goal according to user’s concern [2]. Measure agent identifies quantifiable questions and required software measures. Plan agent identifies the corresponding actions and generates the software measurement plan.

• Analysis group includes recommendation agent and knowledge analysis agent. By tracking the system operational data, recommendation agent provides recommendations for each step of the Goal-Driven process to help users make choices. The knowledge analysis agent mainly helps the knowledge analyst to make decisions of updating system knowledge base. For example, by providing statistic analysis data of system knowledge usage, analyst can make decisions of maintaining and promoting the frequent-used knowledge, and deleting or degrading the seldom-used ones.

3. Ontology Building

Three points should be emphasized before we proceed through the ontology building in ISMS:

• First, ISMS ontology is so-called task ontology [5], which tries to design a better solution for the Goal-Driven process automation instead of a general ontology for software measurement.

• Second, ontology building in ISMS is an engineering activity within the framework of Methontology [6]. By combining the suggested methodology in methontology [6] and Grüninger & Fox’s methodology [7], the ontology building development process for ISMS itself is defined.

• Third, ontology building in ISMS is an evolving process. For one reason, the building of initial ontology is an evolving process itself [6][8]; for another reason, ISMS has the knowledge analysis and updating ability that the quality of knowledge base can be improved gradually.

3.1. Ontology Development Methodology

ISMS ontology development methodology has five steps, including defining ontology purpose, conceptualizing ontology, implementing ontology, evaluating ontology, and updating ontology.

Defining ontology purpose. Both methodologies presented in [6] and [7] suggest defining the purpose of ontology, but in different ways. The second phase of ISMS begins with identifying the quantifiable questions, which should be answered by the required measures and implemented by corresponding actions. From this point of view, the quantifiable questions have the same function as the competency questions defined in Grüninger and Fox’s methodology [7] to specify the purpose of ontology. Therefore, the quantifiable questions that we identified are treated not only as one part of ISMS ontology, but also as the competency questions to specify the purpose of ISMS ontology. Although there might be concerns about the granularity of the detail-designed quantifiable questions, the original methodology suggests that competency questions should not be simple, but “be in a stratified manner with higher level questions requiring the solution of lower level questions” [7].

Conceptualizing ontology. Explicit ontology conceptual model is usually lost in the rush of implementing ontology in a specific representation language [6][8]. Therefore, the semiformal specification suggested by Methontology [6] is chosen to develop the conceptual model of ISMS ontology. The main function of this semiformal specification is to eliminate the gap between knowledge acquisition and implementation, and facilitate communications between the domain experts and ontology developers [6][8]. In addition, the knowledge domain of ISMS is within the software engineering. Therefore, in order to conceptualize the ISMS ontology we selected UML notation together with natural language descriptions, tables and graphs, in a manner similar to the design document in software development life cycle. Using

1034

UML language and semiformal specification in building ontology has been reported in a number of other works [9][10].

Implementing ontology. Implementing ontology is to convert the conceptual model of ontology into a specific representation language [6][10]. Different kinds of languages and tools can be used to implement ontology [5]. In ISMS, we choose to use Protégé 2000 as the edit tool in that the output can be stored in XML, RDF, OWL, even MySQL format [11].

Evaluating ontology. Both methodologies described in [6] and [7] suggest evaluating ontology with respect to the purpose of the ontology. As the purpose of ISMS ontology is defined by competency questions, we can evaluate the completeness and correctness of answering those competency questions.

Updating Ontology. This step is defined particularly to improve the quality of ISMS ontology. Knowledge analyst with the help of the intelligent knowledge analysis agent can update ontology contents according to its usefulness.

3.2. The Conceptual Model of ISMS Ontology

In order to facilitate each step of the second phase of the ISMS project, ISMS ontology is divided into question sub-ontology, measure sub-ontology, and action sub-ontology. Due to the page limits, only their UML diagrams and a short description of each are presented. The concepts with “#” in the UML diagram indicate that they are defined outside the current sub-ontology, but related to the current sub-ontology. The ISMS ontology overview is shown in Fig. 2.

Question Sub-ontology

Measure Sub-ontology

Action sub-ontology

Answered by

Implemented by

Fig. 2. ISMS ontology overview

3.2.1 Question Sub-ontology. Question sub-ontology is designed to facilitate identifying quantifiable questions and related indicators. As is shown in Fig. 3, concept MeasurementGoal# is the output of the first phase of ISMS [1] [2], while Measure# is on behalf of all measures defined in the measure sub-ontology. Entity is the object of interest, and it can be software product, process, resources, etc. [3]. Attribute and sub-attribute is the characteristics of the entity, such as cost, quality, or complexity [3]. Purpose should be characterizing, evaluating, predicting, or improving the concerned attributes of entities [3]. Question focus is the main concern of different purposes. For example, “What percentage of” is the concern of characterizing, while “what is the highest limit of” is a main concern of evaluating. Perspective is the different point of view, including project managers, customers, developers, etc [3]. Limitation is a time or process filter that limit the concern to a particular time range or software

engineering process. Indicator is defined as a display to help address the quantifiable questions [3]. The question is the identified quantifiable question that is the combination of entity, attribute, question focus, and limitation.

Indicator

Perspective Purpose

MeasurementGoal #Measure#

1..n

1

Attribute

1..n1..n

Sub-Attribute1

1..n

Restricted by

Entity1 n1 nhas QuestionFocus

1..n

1

1..n

1Decomposed to

Limitation

Question1..n

1

1..n

1has

0..n 10..n 1Addressed by

1..n

1

1..n

1Answered by

Combined into

1..n

1

1Constructed by

1..n

Fig. 3. Question sub-ontology

3.2.2 Measure sub-ontology. In order to answer those quantifiable questions, measure sub-ontology defines required software measures and their definitions. The identified concepts, relationships, and constraints are shown in Fig. 4.

ListedMeasure

CommonMeasure Attribute#SpecifiedMeasure0..10..n 0..10..n

uses

11..nClassified by

AnalyticMeasure

AnalyticDefinition

1

1

1

1

defined by

Action#

AggregatedMeasure

n1

n1

PureInternalMeasure

1..n

1

Definition

1

1

1

1

Defined by

1

1

DirectMeasure

1..n

1

Implemented by

n

1

n

1

1

1

1

1

Defined by

Question#Unit

Measure

1..n1

Answered by1 11 1is part of

Scale11

11

is part of

1

1..n

1

1..n

1 1..n

1..n 1

1

Defined by

1

Fig. 4. Measure sub-ontology

The main concept measure, which is equal to “metric” as defined in [3], can be categorized into common measure and specified measure:

• Common measure includes pure internal measure and listed measure. Pure internal measure influences certain overall attributes or sub-attributes [12], while listed measure is pseudo-measures that allow users to choose and customize their own concerns. For example, ISO9126 defines six characteristics of quality [13]. By using listed measure, users can customize their measures

1035

of quality concern to reliability and usability instead of all six characteristics.

• Specified measure is classified by attributes and sub-attributes. It includes direct measure, aggregated measure, and analytic measure. Direct measure is the measure that “does not depend on a measure of any other attributes” [12]. Aggregated measure is well defined or popular-used measures derived from more than one measure [12]. Analytic measure is user-defined measures according to users’ own needs and experience.

The main reasons that measure structure is defined in this way instead of commonly used classification by products, processes and resources (see [14] for example) are that on the one hand, Goal-Driven process is more attribute-centered than entity-centered, and on the other hand, ISMS knowledge analysis and updating provides knowledge promoting and degrading ability, such as analytic measure can be promoted to aggregated measure or aborted, and direct measures can be improved to pure internal measures.

Definition of measure defines assumptions, formulas, and use scenarios of measures in order to guarantee data interpretation consistency and completeness [3]. Analytic definition also needs to include user-defined relationships of related measures. Only pure internal measure and direct measure can be directly implemented by action#.

3.2.3 Action sub-ontology. Action is the implementing steps that can guide users to collect the software measure data [3]. As is shown in Fig. 5, three main concepts are identified in this action sub-ontology, i.e., resource, source, and procedure. Resource defines what we need to fulfill the action. It usually includes tools, storage, people, and time [3]. Source defines where we can get the data. For example, data about defects can be acquired from test reports, reviews reports, inspect reports, etc [3]. Procedure identifies data collecting steps, frequencies of measurement, and methods to collect data [3].

is part of

DirectMeasure#

Tools

PureInternalMeasure#

People Time

Source

Storage

Resource Procedure

Action1..n

1

1..n

1Implemented by

1..n

1

1..n

1implemented by

Fig. 5. Action sub-ontology

4. Conclusions

ISMS is a three-phase research project to facilitate the process of automatically generating a practical software measurement plan. In this paper, we mainly present the work has been done in the second phase of ISMS. Based on the output of the first phase, i.e., the set of specific measurement goals, the second phase converts the measurement goals into a software measurement plan that serves as a guide for users to collect and analyze software measurement data. The system architecture is further expanded to facilitate the second phase of ISMS and to meet the requirement of system learning and knowledge base updating. In addition, the ISMS ontology is built to serve as the knowledge foundation for the system. Future extensions of system learning and knowledge base updating has already been considered and designed in ISMS.

References

[1] Tong Chen, and Behrouz H. Far, “Knowledge Representation and Processing in Intelligent Software Measurement System”, CCECE 2004 – CCGEI 2004, Niagara Falls, 0-7803-8253-6/04, IEEE, May/mai 2004.

[2] Tong Chen, “Knowledge Base Creation and Management for Automation of the Goal-Oriented Software Measurement Using Multi-Agent Technology”, University of Calgary, 2004.

[3] R.E. Park, W.B. Goethert, and W.A. Florac, “Goal-Driven Software Measurement – A Guidebook”, CMU/SEI, 1996.

[4] Tong Chen, Behrouz H. Far, and Y. Wang, “Development of an Intelligent Agent-based GQM Software Measurement System”, 1st Int. Conf. on Agent-based Technologies and Systems, Calgary, 2003.

[5] Dieter Fensel, “Ontologies: A Silver Bullet for Knowledge Management and Electronic Commerce”, 2nd Edition, Springer-Verlag Berlin Heidelberg New York, ISBN: 3-540-00302-9, 2004.

[6] M. Femández lópez, A. Gómez-Pérez, and N. Juristo, “METHONTOLOGY: From Ontological Art towards Ontological Engineering,” Proc. AAAI Spring Symp. Series, AAAI Press, Calif., pp. 33-40, 1997.

[7] M. Grüninger and Mark S. Fox, “Methodology for the Design and Evaluation of Ontologies”, Technical Report, University of Toronto, Toronto, Canada, 1995.

[8] M. Femández lópez, A. Gómez-Pérez, and J. Sierra, “Building a Chemical Ontology Using Methontology and the Ontology Design Environment”, 1094-7167/99, IEEE, Jan/Feb, 1999.

[9] X. Wang and C.W. Chan, “Ontology Modeling Using UML”, 7th

International Conference on Object Oriented Information Systems Conference (OOIS’2001), pp. 59-68, 2001.

[10] Ma de los Angeles Martin and Luis Olsina, “Towards an Ontology for Software Metrics and Indicators as the foundation for a Cataloging Web System”, Proc. of the 1st Latin American Web Congress, 0-7695-2058-8/03, IEEE, 2003.

[11] Protégé Education & Development group, “Protégé-2000 user’s Guide”, Stanford Medical Informatics, retrieved at http://protege.stanford.edu/doc/users_guide/index.html, 2003.

[12] ISO/IEC, “Software Engineering-Product Quality Part 3: Internal Metrics”, ISO/IEC TR 9126-3:2003(E), 2003.

[13] ISO/IEC, “Software Engineering-Product Quality Part 1: Quality Model”, ISO/IEC TR 9126-1:2001(E), 2001.

[14] N.E. Fenton and S.L. Pfleeger, “Software Metrics: A Rigorous and Practical Approach”, 2nd Edition, PWS Publishing, ISBN: 0-534-95425-1, 1998.

1036