Supporting Well-Engineered Web Documentation Development – a Multimedia Software Engineering Approach toward Virtual University Courseware Designs

Annals of Software Engineering 12, 139–165, 2001 2002 Kluwer Academic Publishers. Manufactured in The Netherlands.

Supporting Well-Engineered Web DocumentationDevelopment – a Multimedia Software EngineeringApproach toward Virtual University CoursewareDesigns

TIMOTHY K. SHIH [email protected] of Computer Science and Information Engineering, Tamkang University,151, Ying-Chuan Road, Tamsui, Taipei Hsien, Taiwan 25137, R.O.C.

SHI-KUO CHANG [email protected] of Computer Science, University of Pittsburgh, Pittsburgh, PA 15260, USA

JEFFREY TSAI [email protected] of EECS, University of Illinois at Chicago, USA

JIANHUA MA and RUNHE HUANG [email protected] of Digital Media, Hosei University, Japan

Abstract. Distance learning has become a very important mechanism for virtual university operation. Inorder to realize such an operation smoothly, it is necessary to consider distance learning from three per-spectives: administration, awareness, and assessment. We are currently implementing a virtual universityenvironment according to these guidelines. In this paper, we propose part of such a supporting environmentof the Multimedia Macro-University project.1 One of the most important focuses is a Web course develop-ment paradigm. Web documents are increasingly playing an important role in our daily life, as the Internethas become a new medium for communication and commerce. On the other hand, software developmentparadigms were developed to support program construction. However, these traditional paradigms do notcompletely fit the needs of Web document development due to the following reasons. Firstly, computerprograms focus on problem solving, but Web documents focus on information delivery. Secondly, com-puter programs usually have a fixed size, but this is not true for Web documents, because Web documentsare always evolving as if it were a living document. It is therefore necessary to investigate a new softwaredevelopment paradigm for developing Web documents. We propose such a new paradigm and its support-ing environment, as well as software testing/metrics mechanisms for Web documents. The Web documentsdeveloped using our paradigm are stored in a Web documentation database. From a script description, toits implementation as well as testing records, the database and its interface allow the user to design Webdocuments as virtual courses to be used in a Web-savvy virtual library. The system is implemented as athree-tier architecture, which runs under MS Windows.

Keywords: Web documents, multimedia database, active documents, software metrics, software testing,multimedia software engineering, WWW, virtual university

1 Multimedia Macro-University is a joint research project with the participation of researchers from theUniversity of Pittsburgh, USA, Tamkang University, Taiwan and the Hosei University, Japan.

140 SHIH ET AL.

1. Introduction

Distance learning is a possible revolution of future education. With the growing amountof users on the Internet, Web documentation-based course delivery becomes widelyavailable. Distance learning is thus a trend toward future university. In order to facili-tate the teaching and learning environment, Web browsers, audio/video communicationtools, and data conferencing tools are widely developed. We have surveyed a numberof such systems and come out with a suggestion guideline of the integration of virtualuniversity tools. We believe that, a well-prepared virtual university supporting systemneeds to meet the following three criteria:

• Administration criterion. Besides Web document-based course delivery, a virtual uni-versity environment needs to have administration facilities to keep admission records,transcripts, and so on. These administration tools should be available to administra-tors, instructors, and students (e.g., checking transcript information).

• Awareness criterion. Distance learning is different from traditional education. Sinceinstructors and students are separated spatially, they are sometimes hard to “feel” theexistence of each other. A virtual university supporting environment needs to providereasonable communication tools such that awareness is realized.

• Assessment criterion. Assessment is the most important and difficult part of distanceeducation. Tools to support the evaluation of student learning should be sophisticatedenough to avoid biased assessment.

These criteria are difficult to achieve. However, it is possible to design such avirtual university environment to a certain degree of satisfaction. Multimedia Macro-University (MMU) is a joint research project with the participation of researchers fromaround the world. The primary goal of the MMU consortium is to develop technologiesand systems for the use of virtual university. In this paper, we propose the softwarearchitecture for a multimedia-based virtual course system. The architecture supportsmulti-platform due to the availability of Web browsers and the Java virtual machine.We aim to provide a system on the Internet for instructors to design and demonstratelectures. This system serves as a step toward our research goal – virtual university overthe Internet. The primary directive of our MMU systems has four goals:

• Adaptive to changing network conditions. The system adapts to QoS requirementsand network conditions to deliver different levels of service.

• Adaptive to changing user needs. Users are using the system from different perspec-tives. Types of users include students, instructors, and administrators. The systemsupports the demand of various kinds of information delivery from time to time.

• Adaptive to Web-based environment. The system is Web-savvy. That is, a standardWeb browser is the only software required to students and administrators. The in-structors will use our system running on a Web browser in conjunction with somecommercial available software.

SUPPORTING WELL-ENGINEERED WEB DOCUMENTATION DEVELOPMENT 141

Figure 1. System architecture of the MMU project.

• Adaptive to open architecture. A minimal compatibility is defined as the requirementfor the open architecture. Compatibility requirements include presentation standard,network standard, and database standard.

Figure 1 illustrates the system architecture. Components shown in thick line sur-rounded boxes are tools or systems that we had developed. On the instruction designside, we encourage instructors to use the Microsoft FrontPage editor, or an equivalenton a Sun workstation, to design virtual courses. Virtual courses may also be providedvia some Java application programs, which are embedded into HTML documents. SinceHTML and Java are portable languages, multi-platform courses are thus feasible. Aninstruction annotation editor, written as a Java-based daemon, is also running under theJava virtual machine (which is supported by Web browsers). This annotation daemonallows an individual instructor to draw lines, text, and simple graphic objects on the topof a Web page. Different instructors can use the same virtual course but different annota-tions. These annotations, as well as virtual courses, are stored as software configurationitems (SCIs) in the virtual course database management system. An SCI can be a page,

142 SHIH ET AL.

which shows a piece of lecture, an annotation to the piece of lecture, or a compoundobject containing the above. A class administrator performs bookkeeping of course reg-istration and network information, which serves as the front end of the virtual courseDBMS. The implementation of the virtual course DBMS uses JDBC (or ODBC) as theopen database connection to some commercial available database systems, such as theMS SQL server, Sybase, Informix, or Oracle servers. Currently, our system uses the MSSQL server.

On the other side of the system architecture, a student can use an ordinary Webbrowser to traverse virtual lectures. However, some underlying subsystems are trans-mitted to a student workstation to allow group discussions, annotation playback, andvirtual course assessment. We also provide an interface to the virtual course library. Theinterface allows object searching and browsing. These subsystems, again, are writtenas Java-based daemons running under the Internet Explorer or the Netscape Navigator.Help facilities are provided.

This research project, besides providing a prototype system for virtual universityrealization, also focuses on some research issues in multimedia computing and network-ing. From the perspective of software engineering, several paradigms were used in soft-ware development (e.g., water-fall model, spiral model, object-oriented model, etc.).Can these models be applied to multimedia course development? Or, can we refine thesemodels to be used? On the other hand, how do we estimate the complexity of a courseand how do we perform a white box or black box testing of a multimedia presentationare research issues that we have solved partially. From the perspective of ComputerSupported Collaborative Work (i.e., CSCW), the virtual course system maintains thesmooth collaboration and consistency of distributed course designs. A software config-uration management system should allow the checking in/out of course components andto maintain versions of a course. All instruction systems require assessment. The virtualcourse system has methodologies, which support the evaluation of student progress andachievement.

Due to the growing popularity of Internet, Web documents play an increasinglyimportant role in information exchange, electronic commerce, and our daily life. Theimpact is as far as what numerical computation power of electronic computers do toscientific calculations. In the world of numerical computation, software engineeringprinciples for programming-in-the-large provide guidance to software development andtesting. Similar to numerical computation programs, Web documents are programs thatcontain data and methods (such as specific Java applets). Therefore, developing Webdocuments is similar to developing software programs. However, Web documents aredifferent because they focus on different criteria:

• Web documents focus on information delivery while software programs focus onproblem solving. Even Web documents may embed programs to be downloaded byclients. These programs have computation logic less complicated comparing to thosefor scientific calculations.


• Since Web documents contain information to be retrieved by human users, humanfactors are more important in Web documents, as compared to scientific calculationprograms.

• A Web document is a collection of HTML files, with uniform resource locators(URLs) linking these files. The size of a Web document and its linked referencescannot be predicted, because references can be changed due to network configura-tions or update of the referenced content. On the other hand, a program usually has afixed size of executable code.

• Due to the use of inter-station hyperlinks, Web documents are always evolving. A hy-perlink is valid only within a certain period of time. Similarly, the update of an em-bedded program may introduce new bugs. A developing Web document is not robustbecause invalid hyperlinks and changing environment may affect what can be ac-cessed. In contrast, software programs, in general, are relatively stable after severalcycles of sufficient tests.

Traditional software development paradigms such as the Waterfall model andobject-oriented paradigm have their systematic advantages in building complex soft-ware systems. However, due to the differences mentioned above, these paradigms arenot quite suitable for the development of Web documents. For instance, the diagram-ming techniques of structural analysis/design focus on data and control flows, as well asthe task decomposition and information sharing of a large program. These diagrammingtechniques cannot be adapted directly to Web document designs. On the other hand, theobject-oriented analysis/design paradigm focuses on object classification and providesdata abstraction, property inheritance, and reusability. Classification seems not as im-portant in the development of a home page. Current Web document languages, suchas HTML and XML, do not focus on property inheritance and the mechanism of dataabstraction.

Since a Web document is evolving constantly, sufficient testing will be difficult.Instead, testing and maintenance of Web document should be the same activity while thedocument is delivered. A Web document, or its supporting environment, should have theability to correctly update inter-station hyperlinks (including adding new links). More-over, since Web documents are meant to deliver information, the content should be up-dated periodically. Either through the use of dynamic home page or manually changedby a designer, the Web document should always keep the most current information.Thus, the assessment of documentation usability should be an umbrella activity cover-ing all tasks in the development paradigm. Due to these reasons, it is necessary to in-vestigate new development paradigms for Web documents in the framework of softwareengineering and multimedia computing. In this paper, we propose a new Web documentdevelopment paradigm and its supporting environment.

This paper is organized as follows. We discuss issues related to software life cycles,software testing/metrics and multimedia databases in section 2. The proposed develop-ment paradigm is presented in section 3. A system architecture describing the relation-ships of many useful tools to support such a paradigm is also given in section 3.1. The

144 SHIH ET AL.

Software Configuration Items (SCIs), which we used to compute Web document met-rics, are proposed in section 4. In section 5, we discuss a set of metrics criteria and somealgorithms for the automatic generation of test cases. Finally, we give some conclusionin section 6.

2. Related works

Software development is a complicate process. It requires a systematic approach indifferent phrases of the development. Many software development paradigms were dis-cussed [Pressman 1997]. The linear sequential model, or the so-called Waterfall model,is a classical approach, which has a clear boundary between each phrase of the develop-ment. From the analysis of system requirements, to the design of software architecture,implementation, testing, and maintenance, each step has its goal. This paradigm wasused for many years in program development. The prototyping paradigm allows a soft-ware product to be delivered quickly. The customer validates its functionalities, givessuggestions, and requests for a new version. The advantage of this prototyping paradigmensures an early delivery of validated software. The spiral model is also widely adaptedto the industrial world. This model, with different variants, contains four major phrases:analysis of requirements, implementation, testing, and risk analysis. The developmentprocess usually go through several cycles of these four phrases before a verified andvalidate software product can be released. The spiral model has many advantages fromother models.

There are other models, such as the component assembly model, or the object-oriented analysis/design paradigm [Lorenz]. This paradigm relies on object classifica-tion and object reuse. Usually, an object-oriented language and development environ-ment is used. Reusability and object-orientation concepts can be incorporated into thespiral model, as it is done in practice when a software engineer uses such an object-oriented system for project development. Another technique is the formal specificationapproach. The approach uses formal specification languages based on mathematicallogic and aims to generate software from its specification.

Among the different phrases of software development, software metrics and test-ing are important. Many software metrics for procedure-oriented languages based oncontrol flow and data flow factors have been proposed [Dunsmore and Gannon; Hor-rison and Magel]. On the other hand, complexity studies have been attempted to con-sider object-oriented software systems. The authors [Chidamber and Kemerer 1991a, b]have contributed six candidate metrics (e.g., WMC, DIT, NOC, CBO, RFC, LCOM) tothe development and empirical validation of a set of theoretically motivated metrics forobject-oriented design. Others [Li and Henry] proposed three groups of object-orientedmetrics to evaluate the relations between their metrics and the maintenance effort in twocommercial systems. Other researchers [Kim et al. 1994] also proposed complexity met-rics, which can be calculated directly from program codes from two views: static anddynamic complexities.


In studies on multimedia computing related to testing and performance evaluation,the work discussed in [Dilley et al. 1998] analyzes the performance of Web documents.The evaluation factors are mostly related to workload metrics. Another approach [Choet al. 1998] evaluates the ordering schemes of Web document traversal, and proposesstrategies of traversal to increase the overall efficiency of information retrieval. The im-pact of changes of Web documents to the efficiency of retrieval is studied in [Dougliset al. 1997]. The content and semantics of Web documents are found to be the fac-tors that affect the performance of Web traversal, if appropriate caching mechanism isapplied. There are other approaches addressing software testing related to multimediasystems [Riegel 1997; Misic et al. 1998]. Testing processes are defined and mecha-nisms for measuring the quality of multimedia products are discussed in [Riegel 1997].A framework is presented [Misic et al. 1998] for testing distributed multimedia softwaresystems, with some discussions on test case generation and test architecture.

Multimedia database management system design is an important and interestingresearch topic in the community of multimedia computing and networking. In order tosupport the production of multimedia applications, the management of multimedia re-sources (e.g., video clips, pictures, sound files) is essential. For instance, multimediapresentations can be designed as building blocks, which can be reused. To facilitatemultimedia application design, many articles indicate the need of a multimedia database[Chen et al. 1994; Raymond et al. 1994; Rody and Karmouch 1995; Yoshitaka et al.1994]. A multimedia database is different from a traditional relational database in thatthe former is object-oriented while the latter relies on entity relations. Moreover, a mul-timedia database needs to support continuous resource types of large and variable sizes.Due to the amount of binary information that need to be processed, the performancerequirement of a multimedia database is high. Clustering and indexing mechanismssupport multimedia databases are thus important.

The discussion of research issues in multimedia database management systems canbe found in [Raymond et al. 1994]. A distributed database supporting the developmentof multimedia applications is introduced in [Chen et al. 1994]. A mechanism for formalspecification and modeling of multimedia object composition is found in [Little andArif 1990]. The work discussed in [Little and Arif 1990] also considers the temporalproperties of multimedia resources. A database system for video objects is discussed in[Lin et al. 1994]. A content-based querying mechanism for retrieving images is given in[Yoshitaka et al. 1994]. Layered multimedia data modeling [Gerhard and Michael 1995]suggests a mechanism to manage multimedia data.

In addition to the general discussion on multimedia database management systems(MDBMSs), there are other articles that report a similar approach to ours. The work dis-cussed in [Chen et al. 1995] proposes a multimedia data model and a database to supporthypermedia presentations and the management of video objects. Its specialized videoserver with an incremental retrieval method supports VCR like functions for heteroge-neous video clips. The design of multimedia DBMS is from the scratch, which is similarto our approach. The system also supports object composition/decomposition. However,no specific reuse mechanism is emphasized in the discussion. Only an object-oriented

146 SHIH ET AL.

data model was proposed. The system also provides a global data sharing mechanism,including a video and an image collaboration tool, which are integrated with a distributedenvironment.

A multimedia database for news-on-demand applications is proposed in [Ozsuet al. 1995]. The database follows international standards, such as SGML (Standard Gen-eralized Mark-up Language), and HyTime (Hypermedia/Time-Based Structural Lan-guage). Its Visual Query Interface supports presentation, navigation, and querying.A multimedia type system, especially useful for structured text and presentation infor-mation, is also proposed. This database takes an object-oriented approach, which is alsoused by us. Similar to their standardization approach, we follow standard multimediafile formats by Microsoft, which are worldwide used. The work discussed in [Ozsu et al.1995] has a multimedia type system, which provides a limited object composition mech-anism. However, similar to the work discussed in [Chen et al. 1995], no explicit reusemechanism is provided.

The research in [Chen et al. 1994] uses an object-oriented approach to design aclient-server database environment and a multimedia class library to support multimediaapplications. Its graphical object editor based on OCPN (Object Composition Petri Net)allows scheduling and composing of multimedia objects. The implementation uses theRaima Data Manager/Object Manager (a database system) for its storage model, whichis similar to our early approach. However, we re-design the system from the scratch laterdue to some reasons for extension.

CORBA (Common Object Request Broker Architecture) is a specification andarchitecture standard supported by the OMG (Object Management Group). CORBAaims to obtain distributed open architecture and specification through an object-orientedclient-server structure that achieves interoperability among independently developed ap-plications. Based on the CORBA Object Broker standard, the work discussed in [Heikoet al. 1996] proposes a database component of the VORTEL project. The implementa-tion relies on the MOSS (Media Object Storage System) and the AMOS (Active MediaObject Store) projects for underlying support. The distribution of database structureand the interoperability are based on the DEC’s implementation of the CORBA com-pliant Object Request Broker. Another database design following CORBA is discussedin [Ivan and Tobar 1996]. The Multiware database, as one portion of the MultiwarePlatform project, takes an object-oriented approach and adopts an object database stan-dard (ODMG-93) announced by ODMG (Object Database Management Group). Usingspecialized servers for each media type, the prototype uses the Orbix implementation ofCORBA, and the ObjectStore object oriented database system.

3. The Web document development paradigm

A Web document contains three parts: document script, document implementation, anddocument testing records. The script portion of a Web document can be a text file withreferences to describe the requirements of the document. The implementation of a Webdocument usually contains some HTML (or XML) files, which may embed multimedia


information and control programs (e.g., Java Applets). Different portions of a Web doc-ument implementation are connected by URLs within a station, as well as the referencesof the document linked to files located in remote stations. A Web document implemen-tation may have one or more entry URLs. Starting from these URLs, testing records ofthe Web document can be constructed either automatically or manually. The theme ofthis Web document development paradigm is to provide Web document designers a pro-cedure and a supporting environment to develop Web documents. The paradigm focuseson the following issues:

• Stepwise refinement.

• Information consistency.

• Layered Web document scope.

• Automatic document testing.

• Content assessment.

To design a Web document, the designer may start from a draft document andrefine the document toward a final version. Different portions of a document can bestored in a repository for future references. Web documents evolve. We need a facilityto maintain the “changes” of such an evolving process. We also want to maintain theconsistency of information in the process. From a sketch outline, to the implementationof the Web home page, to the testing records of such a home page, information should beconsistent. From the perspective of implementation, each Web document can be dividedinto two parts: the intra-station document and the inter-station document. The formerrefer to the documentation files stored within a single workstation. These files are furtherdivided according to two types of links: intra-directory hyperlinks and inter-directoryhyperlinks. The latter (i.e., inter-station document) are files, which can be retrieved vialocal area networks or the Internet. This layered definition defines the testing scope ofhome page implementations. A testing tool will test Web documents automatically. Thelayered definition is also used in the process of maintenance. The testing tool should beable to check the validity of links from time to time in order to maintain a Web page.With a supporting environment, the Web document designers should be able to assess thedevelopment status of documents with respect to Web document script, implementation,and testing.

A Web document script is similar to the specification of software. However, thescript may include the process of collecting digital multimedia resources, such as scan-ning a picture or recording a MPEG video file. A Web document script can be specifiedin natural languages, or in formal specifications. The script should be realized as HTMLor XML programs, which can also be generated by commercial Web document designsoftware (such as the FrontPage by Microsoft). Testing of these implementations shouldbe carefully conducted, as well as to maintain them. Incomplete implementation or er-roneous connections should be detected and reported.

We propose a Web document development paradigm (see figure 2) based on thespiral model of a typical software development. The traditional spiral model of software

148 SHIH ET AL.

Figure 2. The spiral paradigm of Web document development.

development has four phrases: analysis of requirements and specification, design andimplementation, testing and releasing, and risk analysis. The first two phrases of ourparadigm are similar to the spiral model, except that Web documents have differentfocuses as we discussed earlier. Since Web documents are to deliver information, thecontent is more important than the feasibility analysis of a system, as it is in a softwaresystem. We also incorporate testing in the process of maintenance, since Web documentsshould be tested for obsolete URLs and missing tag objects in its maintenance. Insteadof using risk analysis after testing, Web document assessment should be used in eachand every cycle of the spiral. The assessment is based on several factors, which will bediscussed in section 5.2. The result of content assessment and testing decides whetherthe Web document designer should deliver a copy of the document, or continue to gothrough another cycle in the spiral.

The development activities in the three phrases (i.e., script, implementation, andtesting/maintenance) as well as Web document assessment are based on the status of adocument repository. With the central repository, the paradigm enforces the consistent


change among each step of the development life cycle. Starting from a requirement,the developer designs a Web script. This script can be a text description of the require-ment of a home page, with optional multimedia resources (e.g., picture, sound, or video)as on-line references. Alternatively, the script can be presented in spoken languagesand recorded as audio records. Whether the script is recorded in text or voice, the im-plementation of the Web script is produced, either automatically or semi-automatically.A physical testing is conducted afterward. The Web document will be delivered after anumber of runs in the spiral. If the content of this Web document is suitable, it couldbe delivered. However, the document should be constantly tested and updated. Thedevelopment life cycle of a Web document can be considered at three different levels:intra-directory level, intra-station level, and inter-station level. How to divide documentsat each phrase depends on the range of references or URLs. For instance, a documentscript may refer to a multimedia resource file within the local station, or another Webpage in a remote site. The implementation will use different ranges of URLs with respectto the script. The testing scope of URLs can thus be decided afterward.

The three phrases of the paradigm contain document items, known as SoftwareConfiguration Items (SCIs). These SCIs should be maintained in a layered multimediadatabase. Changing an SCI in one phrase may affect the others. The supporting toolof our paradigm aims to provide a control mechanism to ensure the consistency amongthese SCIs by means of SCI dependency relations.

3.1. Supporting tools of the paradigm

A CASE environment supports the proposed paradigm. In this section, we discuss thearchitecture of the CASE environment (see figure 3). Detailed techniques of the designand implementation of this supporting environment will be discussed in the followingsections. For each phrase of the paradigm, we aim to provide some tools. These toolsare software components in the system:

• Script. A text/sound Script Editor allows the Web document designers to design doc-ument scripts and to collect digital media of various types, including sound, picture,video, and MIDI records. These media data are kept in a resource pool, which can bedirectly accessed by a commercial database server. Document scripts can be writtenas hypertext, with multimedia resource files as on-line references. A document scriptis not a home page. Instead, it describes what should be included in the home page.

• Implementation. For each portion of the Web document script, the designer can usethe FrontPage Web editor by Microsoft, or other similar tools to implement the cor-responding HTML files, which may link to each other or invoke some multimediafiles. The implementation tool is not part of the development of our CASE environ-ment. But, our environment integrates such a tool. Each implementation has someSCIs. SICs at the implementation level can also be used as a base of Web documentmetrics, which is estimated by a metrics tool.

• Testing and maintenance. An automatic testing tool allows the developer to test thecompleteness and soundness of a Web document implementation. Testing is divided

150 SHIH ET AL.

Figure 3. The system architecture of Web document development paradigm.

into at least two levels according to intra-station or inter-station links. For inter-station testing, the user is able to specify a range of URLs to be tested, either bydomain names, or by the depth of references. After the first test is conducted, thedesigner may post the home page. The home page will be tested periodically by thetesting tool to omit the misusage of any obsolete URL.

• Repository. A multimedia database is used to manage SCIs at the three phrases.SCIs at different levels are related. These relations are used in an inference systemsuch that when an SCI is changed, the supporting environment will trigger a updateto other SCIs. SCIs of different phrases are organized in a repository, which is adatabase schema. The physical data will be stored using a relational database server.

4. Software configuration items of Web documents

Software Configuration Items (SCIs) are documentation elements consisting of a soft-ware system. In a typical software development process, SCIs exist in different devel-


opment phrases from specification and design to testing and maintenance. The manage-ment of SCIs is an important task due to two reasons. Firstly, SCIs are evolving fromtime to time. It is necessary to keep track of the changes in a software development suchthat, if required, different versions or variants of the software can be produced. Maintain-ing the history of these changes also helps a system developer to learn from the preciousexperiences. Secondly, in a collaborative development environment, it is important to ef-ficiently divide documents or programs into small SCIs in order to have different devel-opers working on different portion of a project concurrently. The maintenance processof SCIs thus becomes an umbrella activity through the software development process.

From the perspective of Web document development and according to our para-digm, software configuration items are divided into three levels. Each SCI in these threelevels has a number of attributes, as described below:

• An SCI at the script level may contain the following objects:

1. A specification of the Web document, which includes name of the specification,the author’s name, the creation date/time, and the specification main text withoptional audio records.

2. The development status of a Web document, which includes the expecteddate/time of completion, and the percentage of completion.

3. References include file names of multimedia resources, the starting URL of theimplementation SCIs, indices to test records, and indices to bug reports.

• An SCI at the implementation level of a Web document should have:

1. A set of implementation objects, which include a collection of Web documentmetrics items discussed in section 5.1. The set of implementation objects, with astarting URL, will be constructed with respect to a specification.

2. References include an index to the specification, indices to test records, and in-dices to bug reports.

• A test record SCI of a Web document may contain:

1. Test record information, which contains a testing scope (local or global), the start-ing URL of the implementation SCIs, name of the specification, and is a list ofWeb traversal messages.

2. References include references to bug reports.

• A bug report SCI of a Web document may contain:

1. A bug report of each test, which includes name of the development engineer, nameof the quality assurance engineer, a test procedure, and a bug description.

2. Errors and incomplete items, which include bad URLs, missing objects, inconsis-tency, and redundant objects.

152 SHIH ET AL.

The use of references at each level is to ensure the consistent update of SCIs. A setof dependency relations among SCIs at different levels is defined. Based on these rela-tions, if an SCI at one level is changed, its related SCIs at another (or the same) levelshould be changed as well. For instance, if a specification is changed, its correspondingimplementation and testing record could be changed. A regression test should be per-formed with a possible change in a bug report. If an implementation SCI is changed, itis required to check whether the specification should be changed and a regression test isexecuted.

As an example of SCIs, we present the three portions of a Web document, as wellas the design of the user interface of our supporting environment. As shown in figure 4,the Web document script SCI contains the following parts:

• Specification name: DMS’98 Main Page.

• Author: Lawrence Y.G. Deng.

• Date: 6/15/98 16:17.

• Description: The Web document is a call for papers of the DMS’98 conference. Youshould use “title.bmp” as the header of the main page. . . .

• Expected date/time of completion: 6/30/98 12:00.

• Percentage of completion: 98%.

• Multimedia resources: a set of multimedia resource reference icons, as shown inthe figure.

• Starting URLs: http://www.mine.tku.edu.tw/dms98, . . . .

• Test records: a set of testing record references, such as test1010.

• Bug reports: a set of bug report references, such as report1312.

The user starts from the script design phrase of the spiral model shown in figure 2.Requirements of the document script as well as auxiliary information are given in theWeb Script window (see figure 4).

According to the script, the Microsoft FrontPage editor is used to generate theWeb document implementation shown in the Web browser. While the user is designingthe script, he/she can look at some multimedia resources via the resource icons. In theimplementation phrase, the user also needs to provide auxiliary information shown inthe Web Implementation window in figure 5. An implementation SCI has the followingitems:

• Implementation objects: such as hyperlinks, data objects, and add-on control pro-grams.



• Test records: a set of testing record references, such as test1010.



Figure 4. A Web document script and its implementation.

Figure 5. A Web document testing environment.

154 SHIH ET AL.

In the testing and maintenance phrase of the spiral model, each test record has:

• Testing scope: local or global.



• Web traversal messages: windowing messages to perform the test.


A test record is generated with the help from the Web document designer. Withrespect to a test record, a set of bug reports is associated. Each bug report contains:

• Development engineer: Lawrence Y.G. Deng.

• Quality assurance engineer: Timothy K. Shih.

• Test procedure: Click on the “Transportation” anchor. You should see a map of theTaipei city, a subway map, . . . .

• Bug description: The description of transportation is missing. The bus schedule is. . . .

• Bad URLs: a list of URLs that can not be connected.

• Missing objects: a list of objects missing from the implementation.

• Inconsistency: a text description of inconsistency.

• Redundant objects: a list of redundant objects.

The development process may proceed through several cycles in the spiral modelshown in figure 2. As the Web document evolves, the user may start from a Web docu-ment only containing the script portion. The implementation and test records are omittedat the beginning. In the next iteration, the user uses the FrontPage editor to design themain page of DMS’98, and add intra-directory and intra-station hyperlinks. The testrecord has a local testing scope in this cycle. Assessment is conducted to decide thestatus of the document, including checking the percentage of completeness and incon-sistency, to decide whether to proceed with the next iteration. When the user starts toadd inter-station hyperlinks in the next cycle, the testing scope should be changed. As-sessment is then performed again to decide whether to post the DMS’98 main page. Theevolution process of a Web document can be assessed from another perspective. A doc-ument may start from an implementation without add-on control programs. Gradually,the user adds Java applets. Regression tests should be preformed at each cycle. Anotherway to look at the evolution is that the user can start from a Web document without im-age and audio records. The user can add images in the next cycle, and audio records inthe next following cycle, and so on (for video clips). The spiral model can be used indifferent types of Web document evolutions.


5. Web document metrics and testing of Web documents

Software metrics monitor the complexity of programs. Software testing ensures the ro-bustness of a software system. Metrics and testing are correlated since from softwaremetrics items, it is possible to deduce some testing guidelines or automatic procedures.In a software development paradigm, testing is essential since the risk of erroneous soft-ware should always be reduced. However, in the literature of software engineering, littleattention has been considered in metrics and testing of Web documents. In this section,we present some research results in these topics.

5.1. Web document metrics

Web documents, whether they are constructed manually or via some tools, are HTMLor XML files. These files contain structured tags and the associated information, suchas pictures, audio records, or control programs to carry out these media. To estimate thecomplexity of a Web document, the first step is to understand what basic items a Webdocument contains. In general, a Web document may include one or more HTML files,which have the following primitive objects embedded in different varieties of tags:

• Hyperlinks. Links are logical connections among hypertext documents. A link isdivided into two parts: an anchor and a URL (uniform resource locator). An anchorexists as a text-based or bitmapped button. When activated, the Web browser bringsup a new portion of Web document indicated in the destination URL. The new portionand the link could be in the same HTML file.

• Data objects. Data objects could be static or dynamic depends on their mediumcharacteristics. Text, pictures, forms, and rulers are examples of static data objects.Audio records and video clips are dynamic data objects. In addition, attributes, suchas font size or object color, are data objects that can be embedded in tags.

• Add-on control programs. Web documents could be dynamic in many ways. Forinstance, a document may contain animation pictures. Another may contain databasequeries to retrieve information from an on-line remote database server. These eventsare control by some add-on programs. Some examples of add-on programs are Javaapplets, Active X controls, Visual Basic scripts, Java scripts, CGI and Perl Programs.

The structure of a Web document and its navigation sequences are two importantfactors to estimate Web document metrics. From the perspective of documentation struc-ture, Web documents are composed physically from the above primitive objects accord-ing to a tree structure, which contains tags. The sizes of data objects and add-on controlprograms embedded in these tags measure the metrics. On the other hand, from the per-spective of documentation traversal, a directed multi-graph2 with loops3 (i.e., links withorigin and destination of the same node) can be used to model user navigations. Within

2 The reason of using a multi-graph is that the user can have different links jump from one HTML file toanother HTML file.

3 The reason of loops due to bookmarks which jump from a location to another in the same HTML file.

156 SHIH ET AL.

a navigation graph, a node with no incoming edge is a source node. A node without out-going edge is a sink node. A Web document has hypermedia components traversed bynavigation trails. A navigation trail (or trail) contains a number of links, which connectsdata objects and control programs. Note that, a trail may or may not be a circuit. A nav-igation circuit is a closed navigation trail. With respect to an individual Web document,an acyclic full navigation trail is a navigation trail starts at an entrance of the Web doc-ument and ends at a dead end object (or a sink node). A source node must be one of theentrances of a Web document. Otherwise, the source node must be a redundant node tothe document. But, an entrance of a Web document can be any node. Note that, the nav-igation sequences of a Web document compose a multi-digraph with possible multiplesource and sink nodes. It is possible to have two or more links between a pair of nodessince an HTML file4 can contain two anchors linked to the same URL. In our model,the multi-digraph of a Web document is connected. Otherwise, there exist more thanone Web document. The graph model can be decomposed into acyclic full navigationtrails and navigation circuits, which are two categories of traversal behavior. We believethat, these two traversal structures are sufficient to represent all navigation sequences ofa Web document traversal. These two types of structures and their portions are what weneed to consider in our automatic testing procedure. Figure 6 illustrates such a Web doc-ument traversal multi-digraph. The graph has two entrances (a and d, shown in boxes)and four sinks (g, h, j, and k, shown in circles). There are five navigation circuits and 24instances of acyclic full navigation trails:

Navigation circuits Acyclic full navigation trails Acyclic full navigation trails

a–b–e–a a–b–e–i–k (three instances) d–ha–c–f–e–a a–c–f–e–i–k (three instances) d–a–b–e–i–k (three instances)a–d–a a–b–f–j d–a–c–f–e–i–k (three instances)a–b–f–d–a a–c–f–j d–a–b–f–ja–c–f–d–a a–c–g (two instances) d–a–c–f–j

a–d–h d–a–c–g (two instances)a–b–f–d–ha–c–f–d–h

Another consideration of metrics evaluation is the locality of links and objects.Uniform resource locators, according to their destination, can be subdivided into threecategories: bookmarks, local URLs, and remote URLs. A bookmark allows a navigationjump to another location within the same HTML file. Thus the multi-digraph modelmay contain loops. A local URL jumps to a location in an HTML file, which is storedwithin the local workstation (i.e., no network access is required). Local URLs can befurther subdivided into intra-directory level and intra-station level. The navigation ofa remote URL links to an HTML file in a remote workstation. Similarly, a primitive

4 In this multi-digraph model, we treat each HTML file as a node. However, it is possible to further subdi-vide an HTML file into several segments. Each segment is a node.


Figure 6. A sample Web traversal multi-digraph.

object attached to a tag could be a local embedded object or a remote embedded objectdepending on where these objects are stored. Locality of links and tag embedding maynot affect the size of a Web document. But, it significantly affects the performance of anon-line navigation.

We believe that, Web document metrics can be considered at least from the follow-ing four aspects:

• Traversal metrics. From the perspective of human interaction, traversal metrics de-scribe the complexity of a document traversal. The metrics needs to take the numberof navigation trails and cycles into consideration:

TMn = number of URLs × w1+ number of acyclic full navigation trails ×w2

+ number of navigation circuits ×w3,

where w1 < w2, w1 < w3, w1 + w2 + w3 = 1.0.

Note that, the weight of URLs should be lower than those of trails and circuits.

• Data metrics. From the perspective of storage volume, data metrics estimate thestorage volume of a Web document. The sizes of primitive and compound objects areconsidered:

DMs = total size of HTML files ×w4 + total size of media files ×w5,

where w4 > w5, w4 + w5 = 1.0.

Note that, text in HTML files should have a higher weight than media data of thesame size.

• Computation metrics. From the perspective of CPU load, computation metrics es-timate the performance of add-on control programs. Traditional software metricstechniques can be used to evaluate the size or the complexity of these control pro-grams:

CMn = total number of add-on control programs,

CMs = total size of add-on control programs.

158 SHIH ET AL.

• Transmission metrics. From the perspective of network communication, transmissionmetrics estimate how network traffic load affects a Web document. In this case,localities of links and tags are considered. However, transmission metrics work withrespect to documents of an individual station only:

RMn = number of remote URLs,

RMs = total size of remote embedded objects.

Web document metrics are estimated according to number of objects and sizes of objects.As a summary of the above four metrics aspects, Web document metrics can be estimatedas:

WDMn = TMn × ptn + CMn × pcn + RMn × prn,

WDMs = DMs × pds + CMs × pcs + RMs × prs,

where ptn, pcn, prn, pds, pcs, and prs are weights.Metrics weights and items can be used as a base for designing the automatic testing

procedure of Web documents, as we will discuss in the following section.

5.2. Web document testing and maintenance

In section 3, we argue that, testing and maintenance of Web documents should be thesame activity. The reason is due to the constant change of remote URLs or Web docu-ment components. We want to design a test procedure and its supporting environment totake Web documentation scope into consideration. The test procedure has the followingtwo scopes:

• Local testing scope. All URLs and embedded objects within the server where the Webdocument is located should be tested. These URLs and embedded objects could be inthe same directory or different directories. Remote URLs and objects are ignored atthis level. Local testing is feasible all the time since local data can be retrieved easily.

• Global testing scope. URLs and embedded objects in a LAN or the Internet environ-ment are tested. Remote testing is hard to construct at the first time since all remoteURLs need to be traversed. However, after the traversal graph is constructed, test-ing follows. Global testing should be conducted periodically for the sake of Webdocument maintenance. The testing scope can be decided based on the followingstrategies:

1. The depth of navigation trails (to one level of remote URLs usually).

2. The range of network IP addresses.

3. A preference profile, which contains the scope information.

The testing process can be conducted semi-automatically or automatically. Theprocedure should check the following type of errors or incompleteness:

• Bad URLs. Illegal URLs or problematic servers should be located.


• Incomplete designs. Documents under construction, such as missing embedded tagobjects should be high-lighted.

• Inconsistent designs. If an embedded object need an add-on control program, thecontrol program should be included.

• Redundant objects. Data objects not linked by any URLs and control programs notused should be pointed out.

Incompleteness, inconsistency, and redundancy checks help a user to measure thestatus of his/her Web document. Measurements on a document can be collected usingthe proposed Web document metrics. Further Web document testing produces additionalmeasurements on, for example, percentage of bad URLs, percentage of missing tag ob-jects, and so on. These measurements are evaluated at the Web document assessmentstage, which determines whether the next level of Web design should be entered.

The process of checking relies on some graph traversal algorithms, which visitdifferent portion of a multi-digraph. Two levels of checking are considered:

• Level 0 testing. All URLs should be tested. All primitive objects in a Web documentshould be tested, including all data objects and add-on control programs.

• Level 1 testing. All acyclic full navigation trails and navigation circuits should betested. Also, all compound objects in a Web document should be tested.

The above two levels of testing should cover all edges and nodes of a connectedWeb document graph. However, level 1 testing, if successfully conducted, should coverlevel 0 testing since all nodes and links are covered in the collection of all trails andcircuits. Thus, it is important to find trails and circuits in the graph. Before we presentour algorithm, we propose some theorems:

Theorem 1. Adding a new edge to a multi-digraph will introduce a circuit only if nonew node is added. That is, both the origin node and the terminus node of the new edgewere in the original multi-digraph.

Theorem 2. In a multi-digraph, adding a new edge to the graph introduces one or morecircuits if the origin node of the new edge is a descendant of the new edge’s terminusnode. Otherwise, no circuit is created. If the trail on which the new edge is addedcontains multi-edges (i.e., edges with the same origins and terminuses) or the trail isa part of a DAG (Directed Acyclic Graph) with multiple paths, the circuits introducedcontain the permutation of all multi-edges and all trails in the original DAG.

Theorem 3. Adding a new edge into a circuit itself introduces exactly a new circuit. Ifthe new edge has its origin and terminus both belong to a common portion of two ormore circuits, a possible new circuit is created for each original circuit, which shares thecommon portion. If the new circuit created contains multi-edges or edges embedded ina trail which is a part of a multi-path DAG, then multiple permutation instances of thecircuit results.

160 SHIH ET AL.

Figure 7. Adding edge to a multi-digraph.

Theorem 1 is easy to proof since a new edge with one or two dangling nodes (i.e.,nodes with the sum of in-degree and out-degree equals to one) cannot be contained ina circuit. If a new edge added introduces one or two new nodes, at least one danglingnode is created. In theorem 2, according to theorem 1, a new edge added must haveits origin and terminus both in the original graph. Since the graph is directed, an opentrail exists from an ancestor to a descendant, but not the other way around. A circuit(closed trail) results only if we add a new edge from the descendant to the ancestor.Theorem 2 is also illustrated in the graph on the left side of figure 7. The new edge(h, a) is added. Since edge (b, c) (i.e., a multi-edge) has two instances, and the graphis a DAG with multiple paths, there are four circuits introduced (i.e., a–b–c–d–e–g–h–aand a–b–c–d–f–g–h–a, with two variants of each). However, if the new edge (a, h) isadded, there is no circuit created. When the new edge (d, a) is added into the multi-digraph on the right side of figure 7, there are two new circuits created: a–b–c–d–a withrespect to a–b–c–d–e–a, and a–f–g–h–d–a with respect to a–f–g–h–d–e–a. This revealstheorem 3. These theorems are the bases we rely on to construct our trails and circuitsfinding algorithms.

Our two algorithms find the two types of structures of a Web document traversegraph. It is easy to find redundant objects by comparing the nodes and their embeddedobjects with the HTML files and the associated resources available. Based on the result-ing structures, test cases are constructed to check the inconsistency and incompletenessof the Web document. The first algorithm finds all circuits in a Web document graph.The computation is based on a concept of the above theorems. The algorithm uses a DFSor BFS traversal algorithm to construct a spanning tree from the graph. Then, for thoseedges that are in the graph but are not in the spanning tree, the algorithm inserts theminto the graph. In the insertion process, theorems 2 and 3 are applied. The output ofthis algorithm will contain a set of navigation circuits, which will be used in the secondalgorithm.


Algorithm. Find_CircuitsInput: A Web document multi-digraphG = (V ,E)

Output: A set of circuits C

Procedure:

1: Let T = (V ,E′) be a spanning tree obtained from G,using a DFS or BFS traversal algorithm on G

2: Let C = 0 be a set which will contain circuitsLet E1 = E\E′ be a set of edges in E but not in E′

3: For each e ∈ E1 do /* based on theorem 1 */Let e = (a, b)

if a is a descendant of b in T then /* based on theorem 2 */if e is not a part of any circuit c ∈ C thenadd new circuits due to e to C

elseFor each c ∈ C where e belongs /* based on theorem 3 */add new circuits due to e to C

elsedisregard e

add e to T

The second algorithm will produce a set of acyclic full navigation trails. The algo-rithm takes as input the graph and a set of circuits from the first algorithm. The use ofcircuit set is to avoid a circuit in the trail. The algorithm uses a DFS traversal algorithmon each entrance node of the graph. In each step of traversal, if an edge belonging to acircuit is visited, all other edges in that circuit are avoided in the traversal. If a sink nodeis visited, the acyclic full navigation trail is added to the output.

Algorithm. Find_TrailsInput: A Web document multi-digraphG = (V ,E) and a set of

circuits C

Output: A set of acyclic full navigation trails R

Procedure:For each entrance node s in the multi-digraph G

perform a DFS starting on s in the graph G

For each edge e on the trail which DFS proceedsif e is in a circuit c ∈ C then

disregard each edge e′ in c

where the origin of e′ is the terminus of e

If a sink node k is visited thenrecord the trail from s to k and put the trail in R

162 SHIH ET AL.

6. Conclusions

Distance learning, virtual university, or remote classroom projects change the manner ofeducation. With the tremendous growing amount of Internet users, virtual university is astep toward the trend of future university. However, most development of distance learn-ing systems uses the high bandwidth of a network infrastructure. As it is not happeningeverywhere on the Internet to meet such a high requirement, it is worthwhile to investi-gate mechanisms to cope with the real situation. Even in the recent future, with the nextgeneration of Internet, the increasing amount of users will consume an even higher net-work bandwidth. The primary directive of Multimedia Macro-University Consortiumis looking for solutions to realize virtual university. Some of our research results, aspointed out in this paper, adapt to changing network conditions. Using an off-line multi-casting mechanism, we implemented a distributed virtual course database with a numberof on-line communication facilities to fit the limitation of current Internet environment.The proposed database architecture and database system serve as an important role inour virtual university environment. We are currently using the Web document develop-ment environment to design undergraduate courses including introduction to computerscience and others.

The paradigm proposed is based on the spiral model of software development.Web document development may not always follow the evolution path from documentswith intra-directory hyperlinks, to documents with intra-station hyperlinks and finallyto documents with inter-station hyperlinks. Another evolution path is from documentswithout scripts/programs to documents with scripts/programs. A third evolution path isfrom text documents to test/image documents to text/image/audio documents and finallyto text/image/audio/video documents. Regardless of the evolution path, the same spiralmodel is applicable.

Our research also focuses on the integration of testing and maintenance, and theactivity of Web document assessment, with emphases on the realization of a repositoryand its consistency checking. Since Web documents are different from programs, theproposed metrics and testing procedures can be used as a new guideline for Web docu-ment testing. Moreover, there are many important factors to the design of our supportingsystem. From the perspective of object-oriented analysis/design, reusability is one ofthe most important issues. The SCIs proposed in the database should be organized in aneffective manner for reuse. Object reuse mechanism adaptable to the paradigm and data-base can be prototype-based. That is, SCIs at different levels can be organized as groups,which are basic elements of object reuse. Another direction of our supporting environ-ment is collaborative design. The underlying database system should be distributed andshould support concurrent access, as well as version controls.

We are applying the proposed methodology to courseware development in the Mul-timedia Macro-University project [Chang et al. 1998]. The empirical study will track thedevelopment of courseware Web documents to identify SCIs for reuse in the further de-velopment of courseware Web documents. However, the proposed paradigm and mech-


anisms only address a small portion of the need of software engineering methodologiesin multimedia computing.

Acknowledgement

We would like to thank Chuan-Feng Chiu of the Multimedia Information NEtwork(MINE) Lab at Tamkang University, Taiwan, R.O.C., for his implementation of the sys-tem.

References

Anthony, G. et al. (1987), “Reusability Issues and Ada,” IEEE Software, July, 43–51.Bruce, A.B. et al. (1987), “The Reusable Software Library,” IEEE Software, July, 25–33.Chang, S.K., E. Hassanein, and C.Y. Hsieh (1998), “A Multimedia Micro-University”, IEEE Multimedia

Magazine 5, 3, 60–68.Chen, T.C., W.P. Lin, C.A. Wu, and C.S. Shen (1994), “A Client-Server Database Environment for Sup-

porting Multimedia Applications,” In Proceedings of the 18th IEEE Annual International ComputerSoftware and Application Conference (COMPSAC’94), Taipei, Taiwan, pp. 215–220.

Chen, C.Y.R., S.M. Dikran, M.C.S. Chang and L.J. Liu (1995), “Design of a Multimedia Object-OrientedDBMS,” In Multimedia Systems, Vol. 3, Springer, pp. 217–227.

Chidamber, S.R. and C.F. Kemerer (1991a), “A Metrics Suite for Object Oriented Design,” IEEE Transac-tions on Software Engineering 20, 6, 476–493.

Chidamber, S.R. and C.F. Kemerer (1991b), “Towards a Metrics Suite for Object-Oriented Design,” InOOP-SLA’91 Proceedings, pp. 197–211.

Cho, J., M.H. Garcia and L. Page (1998), “Efficient Crawling through URL Ordering,” Computer Networksand ISDN Systems 30, 1–7, 161–172.

Chung, C.M., T.K. Shih, J.Y. Huang, Y.H. Wang, and T.F. Kuo (1995), “An Object-Oriented Approachand System for Intelligent Multimedia Presentation Designs,” In Proceedings of the IEEE InternationalConference on Multimedia Computing and Systems, pp. 278–281.

Dilley, J., R. Friedrich, J. Tai, and J. Rolia (1998), “Web Server Performance Measurement and ModelingTechniques,” Performance Evalutions 33, 1, 5–26.

Douglis, F., A. Feldmann, B. Krishnamurthy, and J. Mogul (1997), “Rate of Change and Other Metrics:A Live Study of the World-Wide Web,” In Proceedings of the USENIX Symposium on Internet Tech-nologies and Systems, pp. 147–158.

Dunsmore, H.E. and J.D. Gannon, “Data Referencing: An Empirical Investigation,” IEEE Computer, 50–59.

Enn, T. (1991), “Three New-Generation Software Environments,” Communications of the ACM 34, 6, 46–59.

Enn, T. et al. (1986), “NUT – An Object-Oriented Language,” Computer and Artificial Intelligences 5, 6,521–542.

Gerhard, F. (1987), “Cognitive View of Reuse and Redesign,” IEEE Software, July, 60–72.Gerhard, A.S. and J.W. Michael (1995), “Presentation Layer Primitives for the Layered Multimedia Data

Model,” In Proceedings of the IEEE 1995 International Conference on Multimedia Computing andSystems, May 15–18, Washington, DC, pp. 231–238.

Gibbs, S. (1991), “Composite Multimedia and Active Objects,” In ACM Object-Oriented Prog. System,Languages, and Application, pp. 97–112.

164 SHIH ET AL.

Heiko, T., M. Ulrich, R. Gunter, and M.W. Klaus (1996), “Distributed Multimedia Databases as Com-ponent of a Teleservice for Workflow Management,” In Proceedings of the 1996 Pacific Workshop onDistributed Multimedia Systems, Hong Kong, June 25–28, pp. 111–117.

Henda, H.B. et al. (1995), “A Reuse Approach Based on Object Orientation: Its Contributions in the De-velopment of CASE Tools,” In Proceedings of the SSR’95 Conference, Seattle, WA, USA, pp. 53–62.

Horrison, W.A. and K.I. Magel, “A Complexity Measure Based on Nesting Level,” ACM SIGPLAN Notices16, 3, 63–74.

Ivan, L.M.R. and C.M. Tobar (1996), “Towards an Architecture for Distributed Multimedia Databases,” InProceedings of the 1996 IASTED/ISMM International Conference on Intelligent Information Manage-ment Systems, June 5–7, Washington, DC, USA, pp. 65–68.

James, F.A. (1983), “Maintaining Knowledge about Temporal Intervals,” Communications of the ACM 26,11.

James, M.B. et al. (1995a), “Reuse Through Inheritance: A Quantitative Study of C++ Software,” In Pro-ceedings of the SSR’95 Conference, Seattle, WA, USA, pp. 47–52.

James, M.M. et al. (1995b), “Cohesion and Reuse in an Object-Oriented System,” In Proceedings of theSSR’95 Conference, Seattle, WA, USA, pp. 259–262.

Kaiser, G.E. et al. (1987), “Melding Software Systems for Reusable Building Blocks,” IEEE Software, July,17–24.

Kim, E.M., O.B. Chang, S. Kusumoto, and T. Kikuno (1994), “Analysis of Metrics for Object-OrientedProgram Complexity,” In Proceedings of COMPSAC’94, IEEE Computer, pp. 201–207.

Laski, J.W. and B. Korel, “A Data Flow-Oriented Programming Testing Strategy,” IEEE Transactions onSoftware Engineering 9, 3.

Li, W. and S. Henry, “Object-Oriented Metrics that Predict Maintainability,” Journal of Systems and Soft-ware 23, 111–122.

Lin, K.F., C.W. Chang and S.Y. Lee (1994), “Design of an Interactive Video Database,” In Proceedings ofthe 1994 HD-Media Tech. and Application Workshop, Taipei, Taiwan, pp. PO2-17–PO2-22.

Little, T.D.C. and G. Arif (1990), “Synchronization and Storage Models for Multimedia Objects,” IEEEJournal on Selected Areas in Communications 8, 3, 413–427.

Lorenz, M., Object-Oriented Software Development: A Practical Guide, Prentice-Hall, Englewood Cliffs,NJ.

Manfred, L. et al., “Software Reuse through Building Blocks,” IEEE Software, July, 34–42.McCabe, T.J., “Design Complexity Measurement and Testing,” CACM 32, 1415–1425.Misic, V.B., S.T. Chanson, and S.C. Cheung (1998), “Towards a Framework for Testing Distributed Multi-

media Software Systems,” In Proceedings of the International Symposium on Software Engineering forParallel and Distributed Systems, B. Kramer, N. Uchihira, P. Croll, and S. Russo, Eds., pp. 72–81.

Ozsu, M.T., D. Szafron, E.M. Ghada, and C. Vittal (1995), “An Object-Oriented Multimedia DatabaseSystem for a News-on-Demand Application,” In Multimedia Systems, Vol. 3, Springer, Berlin, pp. 182–203.

Pemberton, D. and I. Sommerville (1997), “VOCAL: A Framework for Test Identification and Deploy-ment,” IEE Proc. Software Engineering 144, 5–6, 249–260.

Philip, J. (1992), “Supporting Exploratory CSCW with the EGRET Framework Emerging Technologies forCooperative Work,” In Proceedings of ACM CSCW’92 Conference on Computer-Supported CooperativeWork, pp. 298–305.

Pressman, R.S. (1997), Software Engineering, A Practitioner’s Approach, 4th Edition, McGraw-Hill, NewYork.

Raymond, P., M.F. Khan, A. Khokhar, and A. Ghafoor (1994), “Issues in Database Management of Mul-timedia Information,” In Proceedings of the 18th IEEE Annual International Computer Software andApplication Conference (COMPSAC’94), Taipei, Taiwan, pp. 209–214.

Riegel, M. (1997), “Assessment of Multimedia Applications and Systems,” In Software-Ergonomie’97.Usability Engineering: Integration von Mensch-Computer-Interaktion and Software-Entwicklung


(Software-Ergonomics ’97. Usability Engineering: Integration of Human-Computer-Interaction andSoftware Development), R. Liskowsky, B.M. Velichkovsky, and W. Wunschmann, Eds., pp. 263–273.

Rody, J.A. and A. Karmouch (1995), “A Remote Presentation Agent for Multimedia Database,” In Proceed-ings of the IEEE 1995 International Conference on Multimedia Computing and Systems, May 15–18,Washington, DC, pp. 223–230.

Ruben, P.D. (1991), “Implementing Faceted Classification for Software Reuse,” Communications of theACM 34, 5, 88–97.

Ruben, P.D. et al. (1987), “Classifying Software for Reusability,” IEEE Software, July, 6–16.Shih, T.K. (1995), “An Artificial Intelligent Approach to Multimedia Authoring,” In Proceedings of the 2nd

IASTED/ISMM International Conference on Distributed Multimedia Systems and Applications, Stan-ford, CA, August 7–9, pp. 71–74.

Shih, T.K., C.H. Kuo, H.C. Keh, C.T. Fang-Tsou, and K.S. An (1996a), “An Object-Oriented Database forIntelligent Multimedia Presentations,” In Proceedings of the 1996 IEEE International Conference onSystems, Man and Cybernetics, Beijing, China, October 14–17.

Shih, T.K., S.K.C. Lo, S.J. Fu, and J.B. Chang (1996b), “Using Interval Temporal Logic and Inference Rulesfor the Automatic Generation of Multimedia Presentations,” In Proceedings of the IEEE InternationalConference on Multimedia Computing and Systems, pp. 425–428.

Spivey, J.M. (1989), The Z Notation: A Reference Manual, International Series in Computer Science,Prentice-Hall, Englewood Cliffs, NJ.

Staehli, R., J. Walpole, and D. Maier (1995), A Quality-of-Service Specification for Multimedia Presenta-tions, Multimedia Systems, Vol. 3, Springer, Berlin, pp. 251–263.

Uszok, A., A. Miazga, and T. Kazmierczuk (1998), “Universal tool for testing CORBA servers,” Zesz.Nauk. Politech. Sl. Ser. Inf. (Poland) 34, 633–648.

Vazirgiannis, M. and C. Mourlas (1993), “An Object-Oriented Model for Interactive Multimedia Presenta-tions,” The Computer Journal 36, 1.

Will, T. (1988), “Software Reuse Myths,” ACM SIGSOFT Software Engineering Notes 13, 1, 17–21.Yoshitaka, A., S. Kishida, M. Hirakawa, and T. Ichikawa (1994), “Knowledge-Assisted Content-Based

Retrieval for Multimedia Database,” IEEE Multimedia Magazine, 12–21.