NAFIPS 2005 - 2005 Annual Meeting of the North American Fuzzy Information Processing Society

Metrics for OntologiesValerie Cross and Anindita Pal

Department ofComputer Science and Systems Analysis,Miami UniversityOxford, OH, USA

{crossv, palaRiDmuohio.edu

Abstract - The success of the Semantic Web has been linkedwith the use of ontologies on the Semantic Web. Given theimportant role of ontologies on the Semantic Web, the need fordomain ontology development and management are increasinglymore and more important to most kinds of knowledge-drivenapplications. More and more these ontologies are being used forinformation exchange. Information exchange technology shouldfoster knowledge exchange by providing tools to automaticallyassess the characteristics and quality of an ontology. The scarcityof theoretically and empirically validated measures for ontologieshas motivated our investigation. From this investigation a suite ofquality metrics have been developed and implemented as a plug-in to the ontology editor Protege so that any ontology specified ina standard web ontology language such as RDFS or OWL mayhave a quality assessment analysis performed.

I. INTRODUCTION

As the current World Wide Web evolves into theSemantic Web, the human burden of information access,extraction, and interpretation is shifting to more automatedservices that rely on machine processable ontologies.Ontologies explicitly link the form and the content of theinformation. Just as the intended purpose of the World WideWeb was knowledge sharing among humans, the SemanticWeb's intent is to permit sharing of knowledge amongsoftware agents. The extent to which reuse of ontologiescould contribute cost and time savings parallels that obtainedin software reuse [1] because acquiring domain knowledge,constructing a conceptual model and then implementing themodel require a huge effort. Ontologies, however, are datamodels and very different from a software components whichcan be evaluated for correctness based on a given processspecification such as an input/output function [2]. In addition,the Semantic Web is an independent environment with norestrictions on the shared knowledge. As the demand forexploiting the reuse of ontologies grows, there is a need forcriteria or standards to adequately determine the quality of theontologies being made available to both humans and agentsalike on the Semantic Web. Information exchange technologyshould foster ontology reuse by providing tools to automatethe process of evaluating the usefulness and quality of anontology.

A variety of frameworks have been proposed forcomparing ontologies. One of the earlier studies [3] examinedgeneral features referred to as fundamental propertiesconsidered important to the ontology design process andrelevant to ontology reuse. The three properties considered

are formality which ranges from highly informal to rigorouslyformal, purpose or the intended use of the ontology, andsubject matter which is what domain the ontology isdescribing. An objective of our research is to contribute toautomating the process of determining a level of formality ofan ontology. Another earlier and more detailed-oriented study[4]compares ten different ontologies with respect to 28characteristics grouped into the following eight categories:general, taxonomy, design process, internal concept structureand relations between concepts, axioms, inference mechanism,applications and contributions. The study examines alreadyexisting and prominent ontology projects to determine thedifferences and similarities in how basic knowledgerepresentation factors are handled. Of the eight categories, theones concerned with the content of ontologies: taxonomy, theinternal concept structure and relations between concepts, andthe presence or absence of explicit axioms are most relevant toour research into automating the ontology evaluation process.

More recently a method called ONTOMETRIC [5]proposes a taxonomy of 160 characteristics, to be used tomeasure the suitability of existing ontologies. These 160characteristics are categorized into five main dimensions: tools,language, content, methodology, and costs. This methodrecommends that a user examine these characteristics for anontology being considered for use in another project. Againthe content dimension is most relevant with respect to ourresearch. It consists of the following factors: concepts,relations, taxonomy, and axioms.. These are very similar tothree of the eight categories in the Noy study.

This paper presents a brief overview of currentapproaches to assessing the quality of ontologies and describesthe design of asoftware plug-in for the Protege OntologyEditor (protege) that performs an ontology evaluation bycalculating a variety of measures. Section 2 describesontologies and various related considerations when trying toevaluate the usefulness and quality of an ontology. Inparticular, the conceptualization complexity and theexpressiveness of the ontology language are important factors.Section 3 presents a variety of approaches for measuring thecharacteristics and the quality of an ontology, some adaptedfrom the research literature in object-oriented design [6] orconceptual modeling [7]. Section 4 explains the design and theinitial implementation of the ontology evaluation plug-in forthe Protege ontology editor. Conclusions and a discussion offuture planned work are presented in Section 5.

0-7803-91 87-X/05/$20.00 ©2005 IEEE. 448

II. ONTOLOGY OVERVIEW

Although many definitions have been given for the termontology, the most common and simply stated is that anontology is a specification of a shared conceptualisation [8].An ontology specifies a shared vocabulary used to model adomain of interest. This vocabulary describes the type ofobjects and/or concepts that exist, their properties andrelations. Standard relations such as is-a, part-of, and instance-of have predefmed semantics.

A discussion of ontologies can encompass many differentaspects. In this section we focus on two primary ones: itsconceptualization complexity and the expressiveness of itsrepresentation language, both of which are very interrelated.Another consideration is its design methodology for whichnumerous one have been proposed [9]. Although an importantaspect, quality measures during design stage are notconsidered here since the initial focus is on evaluation ofontologies currently available for use by an agent or humanthrough the Semantic Web.

A. Conceptualizaton ComplexitySince the methods of organizing knowledge are diverse

and depend on the required level of detail and logic needed bythe problem domain ontologies may vary not only in theircontent, but also in their structure and implementation. A lackof clarity exists when using various terms to describeknowledge organization methods [10]. In practice, aclassification, directory, a thesaurus, a taxonomy, a semanticnet, a frame system, or a logical model have been referred toas an ontology. This practice has been attributed to the termontology being currently the more fashionable one.Clarification of some of these terms is provided below startingwith classification followed by taxonomy since they buildupon each other. Taxonomy plays a role in describing otherterms .Two prominent ontologies, WordNet and UMLS arebriefly described to firther emphasize the different levels ofconceptualizaton complexity.

A classification systematically organizes objects or itemsinto categories according to selected criteria. The criteria areexternal to the objects. A taxonomy is a hierarchical system ofclassification where the criteria represent structural or internaldifferences. The classification categories are ordered fromgeneral to specific (top to bottom).based on an is-arelationship between categories. A taxonomy has two aspects,its nomenclature and its terminology. The taxonomy'snomenclature is the arrangement of its classes or categories.Its terminology is the words used to label the variouscategories.

A thesaurus is a list of related word groups organized by acombination of taxonomic, ontological and dictionaryattributes. A thesaurus represents taxonomic, ontological, andother kinds of relationships. Categories can come from eithera taxonomy or an ontology. Its entries include synonyms butare not simply a list of synonyms since its purpose is to permitdifferentiating between similar words and selecting theappropriate word. An entry does not define the word.

The purpose of a directory is to permit the user to accessone piece of information using another; basically a list ofassociations between pieces of information. Taxonomic orontology-like relationships may exist in a directory; however,its structure is fairly flat. Finally, an ontology minimallyrequires a finite set of unambiguously identifiable classes andrelationships. One of these relationships is the hierarchical is-arelationship. Classes typically have associated properties butthese are optional [11]

Another major difference in ontologies besides thelevel of complexity is the scope and purpose of their content.Domain ontologies that describe specific fields such asmedicine, are clearly recognized as distinct from upper levelor general ontologies that describe the basic concepts andrelationships referenced when domain information isexpressed in natural language. WordNet, an example of ageneral ontology, was developed by Princeton University andserves as a terminological ontology of English [12]. Freelyavailable, this electronic dictionary organizes nouns, verbs,adjectives and adverbs into synonym sets (synsets), eachrepresenting one underlying lexical concept. WordNet isstructured as a semantic network with sets of synonymousterms, or synsets, constituting its basic organization. Thesynsets differentiate word senses from each other and underlielexical concepts. For example, the word "address" correspondsto two different lexical concepts, one related to making aspeech and the other to designating a location. Lexicalconcepts are linked by a variety of relations such as hypernym(is-a), hyponym (subsumes), antonym, holonym, (has-a) andmeronym (part-of) connections. Thus WordNet has anunderlying taxonomic structure based on the hypernmrelationship between lexical concepts. The noun portion ofWordNet is the most developed part of the network and withinit the subsumption hierarchy (hypernymy/hyponym) makes upover 80% of the links. The top of the noun hierarchy contains11 different abstract concepts that are the separate rootstermed unique beginners, for example, entity ('somethinghaving concrete existence; living or nonliving') or event('something that happens at a given place and time').

The Unified Medical Language System (UMLS) is adomain ontology for the biomedical information field anddesigned by the National Library of Medicine[13].UMLSconsists of a large vocabulary database, the Metathesaurus,containing 800,000 biomedical and health-related concepts,various concept names, and their relationships. But theMetathesaurus is not an ontology in the formal sense becauseconcepts are not fully interlinked. The UMLS SemanticNetwork provides an ontological framework for thoseconcepts by assigning each concept a semantic type defined inthe semantic network. Semantic types are linked by semanticrelations in a tree structure. The UMLS Metathesaurustogether with the Semantic Network represent a biomedicalknowledge resource that attempts to standardize the semanticsof the various terms from various biomedical vocabularies andcapture relationships between those terms, which might beboth within and across vocabularies.

449

The previous descriptions of conceptualizatoncomplexity and example ontologies suggest that an ontologyevaluation tool should measure characteristics of an ontologyto determine its conceptualizaton complexity level and mightmap this level into the most appropriate linguistic descriptionsuch as directory, thesaurus, taxonomy, and ontology.

B. Expressiveness ofOntology LanguageNumerous languages for representing ontologies have

been proposed. These languages differ in not only theexpressiveness but also the level of formality [14]. Sinceontologies differ in their required level of complexity andscope, the languages used to specify them need only be as fulland expressive as required to represent the nuance andintricacy of knowledge demanded by their purpose anddevelopers. An argument for language-dependent evaluationtools for ontologies has been made in order to take intoaccount each languages features [15]. Since the features ofmany ontology language are subsets of those found in theOpen Knowledge-Base Connectivity Protocol (OKBC), weinstead propose that an ontology evaluation tool be based onan OKBC-compatible language. OKBC is a common queryand construction interface for frame-based systems thatfacilitates interoperability (REFERENCE).

The main components of an OKBC-compliantknowledge model are classes, slots (either for a relationship oran attribute in object-oriented terminology), facets andinstances. A class is a collection of objects described byidentical properties. Classes are organized into taxonomy or aspecialization and generalization hierarchy, also referred to asa subclass-superclass hierarchy. The superclass represents ageneralization of its subclasses, the subclass, a specializationof its superclass. Slots are associated with each class and areinherited by the subclasses.. Slots (also known as properties)are named binary relations between a class and either anotherclass or a primitive type (such as a string or a number). Facetsconstrain the values taken on by slots, for example, theminimum or maximum value of a slot. An actual member of aclass is referred to as an instance of the class.

The Protege-2000 ontology editor is based on an OKBC-compatible knowledge model. Classes and the class hierarchywith multiple inheritance are supported. Other supportedfeatures include template and own slots; specification of pre-defined and arbitrary facets for slots, which include allowedvalues, cardinality restrictions, default values, and inverseslots; metaclasses and metaclass hierarchy. In [5], the qualityevaluation of an ontology should include the language inwhich it is specified. A table for the language dimension lists38 characteristics broken down into the following factors:concepts/instances/facts/claims, attributes, facets, relations,taxonomies, axioms, and rules. With the table checklist, ananalysis of Protege's knowledge model shows that it provideshigh to very-high support for the first five factors.

Ill. ONTOLOGY METRICS

Several approaches to evaluating the quality of anontology can be taken depending on the purpose of evaluation.The following examples represent a few of the scenarios forevaluating an ontology. A knowledge engineer might need toevaluate several candidate ontologies from which to choosefor incorporating into another system. An ontology is underdesign and throughout this process evaluations needs to beperformed to verify its "goodness." An agent is trying toeffectively communicate with another agent on the SemanticWeb using a different ontology. Determining the level ofsemantic interoperability could require the agent be able toevaluate the quality and usefulness of the other agent'sontology. The approach taken in our research is that of an"ontology consumer" [16] needing an ontology summarization.Other research efforts have focused on objective measures ofthe "goodness" of ontologies [15]. As suggested by Noy,what could be most useful to such a consumer is an ontologysummarization. Noy proposes two kinds of summaries, a top-level summary, possibly a graphical representation of severalof the top levels in the hierarchy and a hub summary, a listingof the hub concepts, those with the largest number of links inand out of them.

The following describes ontology metrics adapted fromresearch literature in other areas such as conceptual modeling,software development, information systems development, andinformation retrieval. New measures based on the informationcontent of a lexical concept in WordNet[ 12] are also proposed.The suggestions described for consumer evaluation metrics aredefined more precisely. Where appropriate the metrics areseparated into two categories: intensional metrics andextensional metrics. Intensional metrics are calculated basedon the ontology definition itself. Extensional metrics measurethe assignment of actual occurrences of ontological conceptsand how effectively the ontology is used to include the domainknowledge.

A. Size MetricsSize measurements of ontology components ontology are

probably not the most important criteria for evaluating anontology but are included for completeness. The followingsize metrics are included (C - class, P-property, A-attribute, R- relation, ) Note that standard deviation is calculated for allaverages to indicate distribution of values. Some of themetrics do not return a numeric value but instead indicateidentifying information. In the following, Cnt stands forcount, Av for average, and Rng for range. The main approachis to determine various measures and to examine them onboth horizontal and vertical slices of the ontology. Intentionalmetrics are prefaced with "i" and extensional with "e."

Size IntentionaliCnt(C) = the number of classes definediCnt(P) = the number of properties definediCnt(A) = the number of properties that are attributes

450

iCnt(R) = the number ofproperties that are relationsiCnt(F) = the number of facets definediMax(P to C) = max number ofproperties defined for a classiMaxClasses(P to C) = classes with max number ofpropertiesiMin(P to C) = min number of properties defined for a classiMinClasses(P to C) = classes with min number of propertiesiAv(P to C)= Cnt(P)/Cnt(C), similarly for A and RiAv(F to P) = Cnt(F)/Cnt(P)iPer(A of P) = Cnt(A)/Cnt(P)iPer(R of P) = Cnt(R)/Cnt(P)

Size ExtensionaleCnt(Ci) = the number of occurrences of class CieCnt(C) = Es Cnt(Ci) total number of object occurrenceseAvCnt(C) = eCnt(C)/iCnt(C)eMaxCnt(C) - maxij[eCnt(Ci)] and identify eMaxCntClasseMinCnt(C) - mini[eCnt(Ci)] and identify eMinCntClasseCnt(Rj) = the number of occurrences ofrelation RjeCnt(R) = Ej Cnt(Rj) total number of relation occurrenceseAv(R to C) = eCnt(R)/eCnt(C)eAv(Rj to Ci) = eCnt(Rj)/eCnt(Ci) where Rj defined for CieMaxCnt(R) - maxj[eCnt(Rj)] and identify eMaxCntRelationeMinCnt(R) - mini[eCnt(Rj)] and identify eMinCntRelation

B. Structural MetricsSructural complexity metrics are often referenced

with respect to object-oriented design [7] since they arethought to be a possible gauge of external quality featuressuch as understandability and modifiability. The typicalrelation used to structure the intensional ontology is thetaxonomic is-a relation, but its corresponding extensionalontology may use a different relation for taxonomic structure(see the discussion in the next section using the WordNetontology as an example). Paths are determined based on thechosen taxonomic relation.

Structural IntensionaliCnt(roots) = number of roots classes

same as NOR [18]iCnt(classes(root1)) = number of total classes for root.

= number of nodes in tree at root1iCnt(leaves(root1)) = number of leaf classes of rootsiCnt(leaves) = Jj iCnt(leaves( roots))= iCnt(paths)

same as NOL [18]iPer(leaves of classes(root1)) = iCnt(leaves(root1)) /

iCnt(classes(root1))iAv(leaves) = ICnt(leaves)/iCnt(roots)iMinDepth( root.) = minj [depth(leafij)]

and Class((leafij ) and Class(root1)iMinDepth() = mini [iMinDepth((root1)]

and Class((leaf;j ) and Class(rooti)iMaxDepth( root.) = maxj [depth(leafij)]

and Class((leafij ) and Class(rooti)iMaxDepth() = maxi [iMaxDepth( root.)]

and Class((leafij ) and Class(root1)iAvDepth(root1) = (j depth (leafij ))/iCnt(leaves(root1))iMaxAvDepth() = max, [iAvDepth(rootr)]

iMinAvDepth() = mini [iAvDepth(root1)]iAvDepth() = (i 2j depth (leaf1j ))/ iCnt(leaves)

same as ADIT-LN [18]iAvAvDepth() =(E iAvDepth(leaves( rootj)))/iCnt(roots)

The iAvAvDepth (average of the average depth) differs fromADIT-LN and seems more meaningful and appropriate sinceeach root class is associated with a different intensional tree

The following measures examine the taxonomic structureacross hierarchical levels for a given root class root1.All thesecan be summarized for all roots in the intensional ontology.

iWidth(depthik) = number of sibling at depth k for root1iMinWidth(root1) = mink [iWidth(depthik)] and levelsiMaxWidth(root1) = maxk [iWidth(depthi9] and levelsiAvWidth(rootQ)=(EkiWidth(depthlk))/MaxDepth(root1)iMinWidth(roots) = mini [iMinWidth(rooti)] and root1iMaxWidth(roots) = maxi [iMaxWidth(root1)] and rootiMaxAvWidth(roots) = maxi [iAvWidth( root1)] and rootiMinAvWidth(roots) = mini [iAvWidth( rooti)] and rootiAvWidth(roots) = Ei [iAvWidth( rooti)]/iCnt(roots)

The new intensional structural metric proposed usesWordNet's information content (IC) measure on itsextensional ontology[17]. The assumption is WordNet'staxonomy is organized in a meaningful and structured way sothat concepts with many hyponyns (more specific concepts)communicate less information than leaf concepts. The morehyponyms it has the less information it expresses. The IC of aconcept within WordNet is computed as

IC.(c) = log [(hypo(c) +1)/max.]/ log(1/ max.)= 1- log(hypo(c)+1)/log(max.)

where hypo(c) is the number of hyponyms of concept c,i.e.,all its descendants, and max. is the maximum number ofconcepts in the taxonomy. The same assumptions for anintensional ontology, meaningful and structured organizationshould hold so that IC for class C,j for root1 is given as

iIC(Cjj) = 1- log(cntDesc(Cij)+1)/log(iCnt(classes(root1)))where cntDesc(Cij) is the number of descendents of Cij.

This measure can be used to identify the degree of informationcontent level by level within the ontology. The followingmeasures are proposed for each root1

iIC(depth,k) = Lj IC(C,j) for all Cj at depth k for rootsiAvIC(depth,k = iIC(depthik)/iWidth(depth,k)iMinIC(depthik) = minj [iIC(C,j)] for all Cj at depth k for root1iMaxIC(depthiok = maxj [iIC(C,j)] for all Cj at depth k for root1iRngIC(depth,k)) = iMaxIC(depthik) - iMinIC(depthik)iMinIC(rooti) = mink [iIC(depthik)] and levelsiMaxIC(rooti) = maxk [iIC(depthi)] and levelsiAvIC(rootQ)= (Ek IC(depthi,))/ iMaxDepth( root)iMinRngIC(rooti) = mink [iRangeIC(depthik)] and levelsiMaxRngIC(rooti) = maxk [iRangeIC(depthik)] and levelsiAvRngIC(root1)=(EkiRngIC(depthik))/MaxDepth( root1)

451

The use of class information content as a new measure on anontology provides a novel way of examining the ontology forsymmetry and balance across its horizontal slices. The samecalculations performed on a per depth level for IC can also becalculated on a per path or vertical slice of a rooted intentionalontology. The equations are not provided due to spacelimitations.

Another new measures proposed here examine the degreeof detail defined with respect to properties per class iDD(P(Cj)both using a horizontal slice (depth level) and a vertical slice(root to leaf path). Both can be refined separately based onattributes or relations, iDD(A(Ci)) and iDD(R(Ci)),respectively. Due to space restrictions no formulas areprovided but a brief outline for calculation is provided. Foreach class in the ontology, count the number of properties(attributes or relations) defined for it. For each depth in theontology on a per rootj basis find the max, min, and averagenumber of properties present in the classes at the given depth.All the previously summarizations over all depths asperformed on IC can then be done for the degree of detailcomplexity. In addition, the summarization can be performedvertically for each root to leaf path to determine, for example,which path on average contains classes defined with the mostdetail.

All the previously described intensional structuralontology measures may also be calculated for an extensionalontology as long as a relationship can be specified todetermine the taxonomic structure of the extensional ontology.The details of these calculations parallel those provided for theintensional model only they are slightly more complex sincethey introduce another layer into the calculations, counting theactual occurrences. The following are a few examples.

ExtensionaleCnt(rooti) = number of occurrences of rootj (class)eCnt(root) = j eCnt(rooti))eCnt(leaves(rooti(occj))) = number of leaves for occurrence j

of rootseMaxCnt(leaves(rootj(occ)) = maxj [eCnt( leaves(rooti(occj)))]

and occurrence of root class i with most leaveseMinCnt(leaves(rootj(occ)) = minj [eCnt( leaves(rooti(occj)))]

and occurrence of root class i with least leaveseCnt(leaves(root-))= Ij eCnt( leaves(rooti(occj)))

=number of leaves for all roots occurrenceseAvCnt(leaves(rootj)) = eCnt(leaves(rootj))/ eCnt(rootj)

C. Consumer MetricsAs previously discussed, the objective of ontology

consumers is to decide whether a particular ontology fits theirneeds. We formalize the suggestions of consumer summanesfor the top levels and the hubs of an ontology by contributinggeneralized versions of these and additional measures to thiscategory such as bottom level summaries and IC-hubs.

A top n-level summary for an intensional ontology at rootjis proposed by reporting previously defined intensionalstructural metrics for user specified n levels starting from theroot down with 0 < n < iMaxDepth( root.).

For all k from 0 to n, report the following: (a sample ofthe possible structural measures): iWidth(depthok) iIC(depthik),iAvIC(depthio, MinIC(depthk),iMaxIC(depthjk) andiRngIC(depthik)). We also propose here a bottom n-levelsummary be provided for a user-specified n levels up startingfrom the lowest level and moving upward in the taxonomicstructure. The bottom level summary would be performed as:For all k from iMaxDepth(rooti).to (iMaxDepth(root4).- n +1).Note that a similar extensional top and bottom n-levelsummary reports could be done for a particular occurrence(occj) of ( rooti).

The hub summary [16] would provide information onan ontology's hub concepts (could be used both for theintensional and the extensional ontology). Hub concepts arethose with the largest number of links (relationships) in andout of them. In [16] it is proposed that measures similar toGoogle's Page Rank metrics where "page" is replaced with"concept" or class should be added to the suite of ontologysummarization measures in order to determine which conceptsare more important within the ontology. Hubs are to beidentified based on a user-specified required number of linksin and number of links out or both. The user specified cut offvalues used for hub identification are normalized values basedon the count of links for the maximumly linked hub in theontology. We propose a local hub summary report based ontop-level-n summary with rootj replaced by local hubh For allk from depth (local_hubh) to iMaxDepth(local_hubh). Thetype(s) of relationship considered for links is parameteried-sothat we may investigate the numerous ways of weighting thevarious links (relations) that can be used to determine a hub,and ways to change the "link" that is used to determine thelevels or distance from the hub based on using different kindsof semantic distance measures [19].

The hub measure looks at links from the local view of aconcept. These are referred to as local hub measures in oursystem. We propose global hub measures by combining the ICmeasure for a class (concept) with the local hub measure tofurther refine the importance of certain classes (concepts).Again all these intensional measures that operate on theontology definition meta data may also be calculated asextensional measures on the instance ontology.

IV. QUALITY MEASUREMENT PLUG-IN

Numerous measures have been proposed to evaluatean ontology. To implement this suite of measures, we havedesigned a plug-in for the Protege ontology editor, a freelyavailable graphical and interactive ontology design andknowledge base development environment. With its highlyusable interface, it has thousands of worldwide users. Itsscalability and extensibility, from its component-basedarchitecture, permit developers to add new functionality bycreating plug-ins. Some important design considerations areintensional vs extensional metrics, selection of taxonomicrelationship for structural complexity, and root determination.

The plug-in has two parts, intensional metrics over theontology definition and extensional metrics, calculated forinstances of the ontology. The design is very parameterized so

452

that it is easily switched from intensional to extensionalontology. Both of these metrics are needed and the ability todo them separately is important. For example, the WordNetontology definition is very simple consisting of only 10classes while its ontology instance is very complex based ontaxonomic relationship of hypernym /hyponym. Similarlythere may be other ontologies where the definition is complexbut particular instantiations are simple.

Measures for structural complexity are based on the is-arelationship or the super-class/sub-class relationship of theontology definition. This relationship is also known as theinheritance relationship. For extensional metrics, however, theplugin allows the user to select the property to use for buildingthe taxonomic structure for the ontology instance since insome cases it may different. The user needs to specify theparent property and child property if upward kinds ofsummaries are need. The extensional structural metric arethen calculated from the graph built on these parent and/orchild properties.

Finding the root(s) of taxonomic structure can be difficult.For intensional metrics the root for the entire ontologydefinition is for example owl:Things (onotolgies imported intoProtege that are defined in OWL) or in other cases the rootmay need to be determined by finding the class with no parentclass in the taxonomy created using the sub-class property. Forextensional metric, find the root is more challenging since itsintensional structure may be flat but the extensional structuremay be built on a specific relationship found in all instances ofone class. For example, in WordNet the class of lexical-concept contains the hyponym (child) and its inverse hyponym(parent) relationships. To find the instance roots withinWordNet, a search over all instances is performed to findthose that have no entries in its hypernymy (parent) property.

V CONCLUSIONS AND FUTURE PLANS

Many knowledge structures are referred to as ontologies,A progression in complexity from simple lexicons orcontrolled vocabularies, to categorically organized thesauri, totaxonomies where terms are given distinguishing properties, tofull-blown ontologies where these properties can define newconcepts and where concepts have named relationships withother concepts. Metrics that can be calculated in order togauge the complexity level of an ontology both intensionallyand extensionally. Our current implementation metrics isbased on developing a plug-in for the Protege ontology editor.This implementation is being experimentally validated againstwell-known ontologies such as WordNet and UMLS. Futureplans are to investigate the degree of detail metrics withrespect to being able to select specific relationships indetermining the degree of linking between classes for theintensional metrics and instances for extensional. Semanticdistance between the classes (instances) may be used to limitthe range of linking that it is included in the metric. Themeasures researched and implemented for semantic distance[19] may be useful for providing insight into statistics oncoupling distances between classes within the taxonomy.

REFERENCES[1] J.S. Poulin J.S. Measuring Software Reuse: Principles, Practices,

andEconomic Models. Addison Wesley Longman, 1997.[2] N. Guarino, "Toward a formal evaluation of ontology quality," in

"Why Evaluate ontology technologies? Because it works!",IEEE Intelligent Systems, July/August 2004.

[3] M. Uschold; and M. Gringer, "Ontologies: Principles, Methods,and Applications" Knowledge Engineering Review. Vol. 11. N.2. June. 1996.

[4] N. F. Noy and C. Hafier, "The State of the Art in OntologyDesign," . AI Magazine. Fall 1997. p. 53-74.

[5] A. Lozano-Tello and A. Gomez-Perez, "ONTOMETRIC:AMethod to Choose the Appropriate Ontology," Journal ofDatabase Management, 15(2), 1-18, April-June 2004

[6] Lindland, O.J., Sindre, G. And Solvberg, A, 1994. Understandingquality in conceptual modeling. IEEE Software, 11(2), March42-49.

[7] Whitmire, S., Object Oriented Design Measurement, John Wiley& Sons, Inc., 1997.

[8] Gruber, T. R. "A Translation Approach to portable ontologySpecifications", Knowledge Acquisition, 5(2) 1993, p. 199-220.

[9] M. Fermandez M "Overview of Methodologies for BuildingOntologies",.ProceedingsoftheIJCAI'99. WorkshoponOntologies andPSMs. Stockholm, 1999

[10] Reinout van Rees, "Clarity in the usage of the terms ontology,taxonomy and classification," Construction Informatics DigitalLibrary, http://itc.scix.net/data/works/att/w78-2003-432.content.pdf

[11] McGuinness, D. L. ".Ontologies come of age".In D.Fensel, J.Hendler, H. Lieberman, and W. Wahlster, ed, Spinning thesemantic web: bringing the world wide web to itsfull potential.MIT press, 2002.

[12] G. Miller, "WordNet: a lexical database for English,"Com7munications of the ACM 38 (11), 39 - 41, 1995

[13] B. L. Humphreys, and D. A. Lindberg, "The UMLS Project:Making the Conceptual Connection between Users and theInformation They Need," Bulletin of the Medical LibraryAssociation 81(2): 170, 1993.

[14] A. Gomez-Perez. and 0 Corcho, 0. "Ontology languages forthe Semantic Web" IEEE Intelligent Systems and TheirApplications 17(1): 54- 60, 2002.

[15] A. Gomez-Perez, "Evaluating Ontology Evaluation," in "WhyEvaluate ontology technologies? Because it works! ", IEEEIntelligent Systems, July/August 2004.

[16] N.Noy, " Evaluation by Ontology Consumers," in "WhyEvaluate ontology technologies? Because it works!", IEEEIntelligent Systems, July/August 2004.

[17] N. Seco, T. Veale, and J. Hayes, "An Intrinsic InformationContent Metric for Semantic Similarity in WordNet," ECAI2004: 1089-1090, 2004.

[18] Yao, H., Orme, A.and Etzkorn, L 2005, "Cohesion Metrics forOntology Design and Application," .1ournal ofComputerScience 1(1): 107-113.

[19] V. Cross and Y. Wang, "Semantic Relatedness Measures inOntologies Using Information Content and Fuzzy Set Theory"Proceedings of the 2005 Fuzz-IEEE Conference, to appear,2005.

453