Case-based reasoning for contractor prequalification

S. Thomas NgDepartment of Building and Real Estate, Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong.

Nigel J. SmithProfessor of Construction Project Management, Department of Civil Engineering, University ofLeeds, Leeds, LS2 9JT, United Kingdom.

R. Martin SkitmoreHead, School of Construction Management, Queensland University of Technology, GardensPoint Campus, 2 George Street, GPO Box 2434, Brisbane, Q 4001, Australia.

Abstract: The contractor prequalification process is characterised by the lack of an universally accepted system and appropriate quantitative and qualitative information. This has led to the development of a number of proprietary prequalification systems together with an over-reliance on human judgement for assessment in practice. To improve the reliability and objectiveness of decisions being made, prequalification needs to be carried out on a more rational basis. An emerging technology in artificial intelligence, namely case-based reasoning (CBR), appears to have high potential to satisfy the specific characteristics of the prequalification domain. The aim of this paper is to demonstrate, through the development of a prototype system, the practicality and suitability of CBR approach for prequalification.

Keywords: Case-based reasoning, contractor prequalification, experiential knowledge.

1. INTRODUCTION

Contractor prequalification is a widely used process intended to ensure a pool of competitive, competent and capable contractors from which tenders may be sought. Despite their strategic significance, existing prequalification processes have their limitations. Not all are fully structured and the formulation of decision criteria depends on human judgement (Ng, 1997a). The procedure for narrowing down the number of contractors to a short-list is usually carried out on an informal basis (Merna and Smith, 1990). The information concerning contractors' attributes consists of both quantitative and qualitative types (Ng, 1996), while the assessment methods used for appraising qualitative information require a predictive judgement of the experts (Nguyen, 1985). As a result, the current practice of prequalification does not guarantee the selection of able and willing tenderers. This can have a significant effect on the successfulness of construction projects.

Attention to date has been focused on making prequalification more rational in order to improve the reliability and fairness of the decisions. Latham (1994) recommends the use of a centralised computer-based system to standardise the prequalification tasks among the public clients. A Data-Base Management System (DBMS) has been developed for prequalifying contractors in practice (Department of Environment, 1992). Paradoxically however, the use of DBMS not only fails to fully address the existing problems of prequalification, but also increases subjectivity by restricting the range of decision criteria to that which might not fully reflect the specific

requirements of the client and project.

Research efforts have been diverted to the development of Knowledge-Based Systems (KBS) (Russell and Skibniewski, 1990; Ng and Skitmore, 1995) which, although designed to mimic the problem solving process of experts, are weak in modelling ill-defined domains (Riesbeck and Schank, 1989). This is a particular problem in prequalification since the construction project requirements and client objectives are dynamic in nature and it is therefore difficult to specify different sets of rules to cover every situation. Case-Based Reasoning (CBR) offers a potential solution to this as it is particularly suited to domains that are not completely understood or when the concept is open-ended (Kolodner, 1993).

To demonstrate the practicality and suitability of the CBR approach for prequalification, a prototype system - EQUAL (denotes Expert QUALifier) - has been developed and evaluated. In this paper, the architecture and key features of EQUAL are described.

2. CASE-BASED REASONING

CBR is an emerging technology in artificial intelligence that "solves new problems by adapting solutions that were used to solve old problems" (Riesbeck and Schank, 1989: 25). This approach offers a paradigm that is close to the way people solve problems (Barletta, 1991) since, in practice, "human experts are not systems of rules, they are libraries of experiences" (Riesbeck and Schank, 1989:15). When dealing with problems in the real world, people are often reminded of a previous similar problem (Schank et al, 1994). Reasoning by reusing or modifying experiences is, therefore, a powerful and frequently applied paradigm for human problem solving (Aamodt, 1990).

In CBR, a 'cases' are defined as "contextualised pieces of knowledge representing an experience" (Kolodner, 1993:13). Cases represent specific knowledge for specific situations. They make explicit how a task is carried out and how a piece of knowledge is applied or what particular strategies for accomplishing a goal were used. In this sense, a case represents the experiential knowledge of the experts in itself. This allows the inference of the CBR systems to be based directly on previous cases (Kolodner, 1987).

The mechanism of CBR is shown in Figure 1. The previous experience of decision makers is extracted and stored as cases in the case-base. Given the details of a new case, the CBR system searches its case-base for an existing case that exactly matches the input specification. The solution of the retrieved case is used to solve the problem without further modification if the new and retrieved cases are the same. If, however, there are no identical cases, the system retrieves the cases that best match the input situation. Since the best matching cases may not properly reflect the new problem, the solutions of the retrieved cases may need to be adapted before they become useful. The new solution is used to solve the current problem, and it is then stored in the case-base to improve the system.

Figure 1 Mechanism of case-based reasoning

Riesbeck and Schank (1989) have used graduate admissions as an example to illustrate that some real world problems have no right answers, and no one will know if they have made the right choice. This illustration parallels the construction prequalification domain. They suggest that, although the decision makers can use rules of thumb (such as "always take the student with the

highest GCSE score"), the actual decision-making process usually depends on a 'gut-feeling' of the situation as a whole. Thus, decision-makers can cite arguments on both sides of the issue and then make a choice that seems best at the time. Such reasoning often depends upon memory.

In contractor prequalification, decision makers equally rely on experiential knowledge to determine the decision criteria and assessing the capability of contractors. Empirical studies by Ng (1996) indicate that clients select the decision criteria by referring to similar prequalification systems designed for the previous projects (see Figure 2). If the recalled systems were developed according to the same characteristics as the present client and project, the same decision criteria could be employed. If not, the decision maker could adapt the criteria to suit the particular circumstances of the present client and project.

Figure 2 Decision cycle for criteria formulation

3. ARCHITECTURE OF EQUAL

The CBR approach has been adopted for developing a computer-based prequalification system - EQUAL. EQUAL is a prototype CBR system designed to perform the prequalification tasks. It consists of five interrelated modules: criteria formulation, screening and reviewing, overall suitability and final scoring, finance, and performance. In addition, system input and output are provided to the users for interrogating any of these modules. Each of the five modules is designated for storing and evaluating various prequalification data. They are also designed to interact with one another.

Figure 3 shows the architecture of EQUAL and the interactions between its modules. The five modules are represented as grey-shaded boxes. The white boxes and cylinders within each module denote the processes or data stores respectively, while the boxes as shown in the system input and output symbolise the information required and generated by the system.

Figure 3 Architecture of EQUAL

The system originates from the criteria formulation module where a set of appropriate criteria for the subsequent assessment modules is determined. The user is asked to define the client, project and prequalification objectives through the system input. Based on this information, the module searches its case-base and retrieves a case (prequalification system) that can reflect the stated objectives. If the retrieved case was based on different objectives, adaptation may be needed. The prequalification system is adapted, and this solution is stored in the case-base for future use. The recommended criteria are reported to the user as an output.

The user is required to enter the necessary contractors' information into the screening and reviewing module when a contractor applies for inclusion. In addition, the year of the latest annual accounts of the contractor has to be specified by the user. This is used for retrieving financial information, which will have already been entered into the finance module by the staff in finance department. The financial information is transfered to the screening and reviewing module. All information relevant to the screening or reviewing process (depending on which prequalification stage it is being used for) is assessed according to the decision rules. An intermediate result is generated and passed to the overall suitability and final scoring module. This result determines whether or not a contractor should be included for further assessment. The reasons for rejecting or excluding a contractor are reported to the user through the system output.

Finally, the contractors are assessed by the overall suitability and final scoring module. This module co-ordinates with and obtains information from the screening and reviewing, finance, and performance module. Any further details about the contractors are entered by the user through the system input. When all the necessary information is gathered by this module, the assessment can be carried out. For the overall suitability assessment, contractors who have similar technical, managerial and financial capabilities are retrieved by EQUAL. Their performance is reported to the user. Based on the performance of the applicant and the retrieved contractors, the system recommends an appropriate category and size of work for the new applicant. The decisions regarding the approved category and size of work are reported to the user. At the same time, the information of the new contractor is stored for future reference. When this module is used for the tender listing process, a hypothetical case is generated to represent the ideal contractor for the specified project type and amount. The hypothetical case is then used to identify how good a contractor is comparing with the ideal contractor (the hypothetical case) for the project. Contractors on the appropriate approved list will receive a similarity score. Contractors with highest scores are reported to the user via the system output.

The reasons for adopting a modular structure were to satisfy the actual functional requirements of the users. Contractors' information is usually maintained by different divisions across the client organisation. For instance, the finance department is responsible for the financial details; the project team is required to provide the project information; whereas the rest of the information is maintained by the contract team. Therefore, it would be more efficient if separate modules were developed for data maintenance. The use of modular structure can also improve the data security and integrity. Shimazu et al (1993) have asserted that a CBR system cannot be used for highly confidential information in the absence of security control. By controlling the access of users to a particular module, such as restricting the project team to interrogate the finance module, sensitive or confidential information can be protected against unauthorised retrieval.

While it is necessary to divide the case into modules, it is important to be able to reconstruct the case. To enable the structure of the case to be preserved, links must be set up between the relevant modules. The principle of linking the modules (case-bases) of EQUAL is similar to that in a relational database management system. In EQUAL, the cases in each module were labelled and stored according to the name of contractors. A link was then created by setting up a pointer to another module. Since the name of cases for both modules were common, information could be transfered from one module to another by simply referring to the name of contractors.

The architecture of EQUAL was represented and developed using a PC-based CBR shell ReMind(tm) version 1.0. The detailed design of each module in ReMind(tm) is described in the following sections.

4. CRITERIA FORMULATION MODULE

This module aims to determine a set of decision criteria that best reflects the organisational, project and prequalification objectives of the client. Appropriate criteria are formulated by reusing and adapting (if required) the prequalification systems of other similar contracting organisations. When developing this module, three issues were considered. First, the module must be flexible enough to cover a range of unusual situations even if similar cases do not exist in the case-base. Second, the

module should be able to retrieve the relevant cases from the case-base. Third, the module should be capable of generating a new solution when the retrieved cases cannot fully reflect the specific objectives of the client.

Case Representation

Aamodt and Plaza (1994) have pointed out that the problem of case representation is to decide what to store in a case and to find an appropriate structure for describing case contents. For this module, the case features included the category and size of work for which an existing prequalification system was used; the type of organisation for which it was developed; and the client, project and prequalification objectives the prequalification system was based upon Russell and Skibniewski (1988). These features constituted the problem part of the case while the solution part of the case was represented by the criteria and sub-criteria adopted in the prequalification system.

Since a number of sub-criteria could be included in each prequalification system, representing each of these sub-criteria as a unique feature would increase the size of the case-base. This can affect the efficiency of the system as more memory may be needed for case storage, and a longer time may be required for case input, retrieval and adaptation. To avoid this, the decision criteria and sub-criteria were represented as a hierarchy of symbols.

As shown in Figure 4, the hierarchy was first split according to the two main prequalification stages, that is the "standing list" and "tender list". Branching off from these two parent categories were the relevant prequalification processes, including screening, overall suitability assessment, reviewing, and final scoring. Decision criteria pertinent to each of these processes were then defined. The lower level of the hierarchy was the decision sub-criteria.

Besides reducing the size of case memory, the use of symbolic hierarchy for representing decision criteria and sub-criteria has two further purposes. First, additional categories can be appended to the tree if necessary. Such flexibility is essential to the users as new criteria and sub-criteria may emerge when cases are added to the case-base. Second, criteria or sub-criteria of one parent category can be related to another through the multiple parentage facility of ReMind(tm) if they are common to both. In Figure 4, criteria pertinent to the overall suitability assessment were linked to the final scoring process. The linkage at an intermediate level, such as financial capability or technical capability, implies that all corresponding sub-criteria below the hierarchy become applicable to the final scoring as well. This not only improves the clarity of the hierarchical tree, but also reduces repetition during the case representation stage.

Figure 4 Symbolic representation for decision criteria

As mentioned at the beginning of this section, a major consideration for this module was flexibility. That means the module should not be restricted to certain common types of organisation only. This is particularly important during the early stage of implementation while the number of stored cases is relatively small. To resolve this problem, a symbolic hierarchy for the types of organisation was created (see Figure 5). The types of organisation were classified into three generic categories namely private, public and quasi-governmental. Each category was split into a number of sub-categories or even further sub-categories. In this way, a general-to-specific hierarchy was created. Although the hierarchy illustrated in Figure 5 may not reflect all the possible categories of organisation, further categories may be added later.

Figure 5 Symbolic representation for types of organisations

The purpose of creating such hierarchy is to guide the system during similarity searching. Consider the district council and the city council, since they were both classified as local government organisations, they would be considered by the system as similar. If a new case was district council, while there were no previous cases of a district council in the case-base, county council may be retrieved as a close matching case. The multiple parentage facility in ReMind(tm) also allows unusual generic categories to be compared with the existing cases. As illustrated in Figure 5, the quasi-governmental client was defined as the second generic parent of non-commercial, central government and local government organisations. When no quasi-governmental cases are available in the case-base, the system will regard its "children" as the next most similar categories for similarity matching.

Indexing and Retrieval

The criteria are generated in stages. The module first determines the suitable criteria for the screening and reviewing process. The criteria for the overall suitability and final scoring process are then compiled. After the initial formation stages, a set of criteria is composed. These criteria are then subject to final refinement. During this stage, historical prequalification systems are retrieved according to the organisational, project and prequalification objectives. The basic criteria are modified to reflect the requirements of the client and project.

To ensure suitable cases are retrieved at the relevant stages, different "views" were created in ReMind(tm). The purpose of developing a different "view" is to facilitate multiple retrievals using the same case-base. When a new "view" is created, the weighting for similarity matching will be set to zero and the adaptation settings and formulae will be ignored. The use of various "views" enabled a new similarity matching scheme to be created for each retrieval stage. More importantly, only those cases that are relevant to a particular stage would be declared as stored cases for case matching and retrieval. This eliminates the possibility of irrelevant cases being retrieved.

Kolodner (1993) has postulated that the ability to retrieve relevant cases from the case-base depends on the appropriate selection of retrieval mechanism. Nearest neighbour retrieval was adopted in this module as the number of prequalification systems (cases) available did not justify the use of an inductive approach. With nearest neighbour retrieval, the features should be indexed for similarity matching. Since there was no exact basis for statistical analysis, the indices were derived from the domain expert by critiquing the intermediate prototype. Miller (1984) has described critiquing as discussing the pros and cons of the proposed approach as compared to alternatives that might be reasonable or preferred. A prototype, with indices assigned by the author based on engineering judgement, was presented to the domain expert. A domain expert was then asked to critique and propose alternate indices for the next prototype. The indices were adjusted according to the expert's recommendations.

Adaptation

The biggest challenge within CBR is to apply the solution of an existing case to a new situation that is a close, but not an exact match (Kass, 1989). In this module, three types of adaptation strategy are provided for the users. They are the null adaptation, critic-based adaptation and generalised adaptation. Null adaptation could be used when the retrieved case is exactly the same as the new one. The user can simply copy the solutions of the matching case to the new case without any changes. When the retrieved case is not exactly the same as the new case, critic-based adaptation could be used (Brown and Lewis, 1993). During the adaptation, the system will prompt the user to accept the solutions of the retrieved case. If the users wish to make changes to any solutions, they can simply jump to the next adapting field without

accepting the recommended solution. The user will then be allowed to enter an appropriate solution according to the different circumstances between the new case and the retrieved case.

Finally, a generalised adaptation strategy is provided in this module. As discussed in the previous section, some of the clients may share the characteristics of both public and private clients. Examples of these are the quasi-governmental clients. For these clients, adaptation from either the private client or public client, no matter how closely matched they are, may not properly meet their distinct characteristics and requirements. For this reason, adaptation from two or more cases may be required. Generalised adaptation is to combine the outcomes of the cases. Currently, the user is allowed to select up to two cases for generalisation. Adaptation formulae were developed in ReMind(tm) to perform the generalisation function. Such formulae allow the module to work out the combined solutions for the new case.

5. SCREENING AND REVIEWING MODULE

The purpose of this module is to eliminate the inherently unsuitable contractors with the reasons for rejection or exclusion being provided for the decision makers. The desire to incorporate the explanation facilities justifies the use of decision rules. The decision rules were developed in the formula editor of ReMind(tm).

Representation of Rules

The formula editor allows decision rules to be created diagrammatically by combining the data fields and various formula functions provided. A typical decision rule namely "R2-FS: Turnover" is presented in Figure 6. The rule was to determine if the turnover of the contractor is greater than or equal to three times the applied price range. If it was not, a statement reads as "insufficient turnover ..." would be returned as the answer for this rule. The upper part of each rectangular box in Figure 6 shows the result during the assessment. For instance, "False" in the ">=" box represented that, after the evaluation, the turnover of the company was not found to be greater than three times the applied price range. This approach was used to develop the decision rules for this module.

Figure 6 A typical rule in the screening and reviewing module

Although the graphical formula editor is easy to understand and use, representing rules diagrammatically with this editor becomes unmanageable when they get more complex. More importantly, since some of the rules for the screening process could be applicable to the reviewing process, representing rules in different levels of detail was considered as a more efficient strategy. In this module, the rules were represented in four levels as shown in Figure 7. Here, financial stability is used as an example to illustrate the structure of rules in this module.

Figure 7 Decision rules represented in four levels of abstraction

The top level (Level 0) included rules for identifying the global results for the screening and reviewing processes. By considering the results of individual criteria at the next level, the rules would recommend if a contractor is eligible for further assessment, or if any constraints should be imposed. For example, if the contractor did not satisfy the requirements of financial analysis, a recommendation for rejection would be suggested.

Rules at Level 1 were criteria-specific. The rules in this level were derived from the outcomes of the rules generated in Level 2. The decision structures of the rules were based on the decision trees. The results of this level could be used to provide the detailed justifications for failing a contractor.

To determine the financial stability of a contractor, a number of detailed financial tests, such as the turnover analysis and financial ratios analysis, need to be performed (see Figure 7). Level 2 contained rules for carrying out more specific tests. Rules at this level were normally specific enough to produce results for the upper levels. However, there were some circumstances where a further level of rules (Level 3) might be inevitable. Ratio analysis is a good example of this. The reason was due to the fact that a number of specific tests, such as profit margin, current ratio, and asset turnover, were involved in the analysis. The results generated at the third level could be useful for other rules in the upper level, such as trend analysis in this case.

In ReMind(tm), the rules created at a lower level can be connected to and manipulated by the rules at the higher level. The principle of this can be illustrated by Figure 6, where the "FS: Turnover (-0)" was a formula field (as distinguished by a small rectangle box at the bottom right hand corner of the tile) to collect information from another case-base. The details of this formula are hidden unless the developer opens it for viewing. This approach enables a complex rule to be developed in a more manageable way.

6. OVERALL SUITABILITY AND FINAL SCORING MODULE

The role of this module is to determine the suitability of contractors for inclusion into the applied standing list and to propose the tender list for a project. The approach used for determining the overall suitability of contractors was based on the proposal of Kolodner (1987). Kolodner has suggested that when a failed case is recalled by the decision maker during the assessment, the previous solution should be adapted to avoid similar errors being repeated. Put into the prequalification context, bad solutions regarding the approved category and price range can be reflected by the poor performance of the approved contractors. In regard to which contractors should be included in the tender list, an assessment was based on the degree of matching between the approved contractors and the hypothetical ideal contractor for the project.

Case Representation

A major concern for this module was how to manage a large amount of features systematically. Apart from the information essential to the overall suitability assessment, further contractors' information must be imported to this module for the final scoring process. The problem was

compounded by the existence of causal relationships among these features. To allow the causal relationships to be represented effectively, a qualitative model as illustrated in Figure 8 was created through the Q-model editor of ReMind(tm).

Figure 8 Causal relationships of case features and decision criteria

In this model, the case features were reassembled to the thirty-five predetermined decision criteria. The causal relationship was defined diagrammatically in the Q-model editor. The example in upper part of Figure 8 shows that the five case features have a positive causal relationship to an intermediate outcome namely "financial stability (overall suitability)", and this together with the "financial stability (ave)" (that is the average score for financial performance), in turn, has a positive relationship with the decision criterion financial stability. Besides positive relationships, an inversely proportional relationship can also be represented in the Q-model. The second half of Figure 8 shows that the number of accidents has a negative relationship (denoted by a negative sign) to the health and safety factor of the contractor. Some features might have a more significant relationship to the decision criteria than the others. To represent the significance of the features, weightings were assigned to the links. These weightings were derived through expert's critique.

Indexing and Retrieval

The initial retrieval for both the overall suitability process and the final scoring process was carried out by template retrieval. Different templates were set up according to the category and size of work. Contractors who were not approved for a particular category or size of work would be eliminated during this retrieval process.

Contractors who had been retrieved by template retrieval were then assessed according to the nearest neighbour mechanism. The indices used for nearest neighbour mechanism were based on the information collected from an earlier study (Ng, 1996). However, the user is allowed to modify the indices as necessary. Maher and Balachandran (1994) have claimed that the use of flexible indices can increase the robustness of case retrieval in real world domains. The selected indices would then be used for computing the similarity scores.

Adaptation

Adaptation might be required during the overall suitability process. Parameterised adaptation was provided in this module. Adaptation rules as proposed by Bailey and Smith (1994) were developed to guide the adaptation process. The adaptation was based on the performance of the new contractors and the similar contractors. The rules were that if the overall performance of the new contractor was below an average of 0.5, the contractor would be rejected. For a contractor who has an overall performance of over 0.5, the performance of the similar contractors would be examined. If the performance of the similar contractors were above 0.5, the new contractor might be included on a probationary basis.

7. OTHER MODULES

Although the three modules as described above can be regarded as the main components of

EQUAL, the significance of the performance module, finance module and system input and output should not be overlooked.

7.1 Performance Module

The performance module was developed to store the performance-related information of the contractors. Most information relevant to this module was fuzzy or symbolic in nature (Ng, 1997b). This kind of information was represented symbolically in ReMind(tm) through the symbol editor. Symbolic values, such as "good", "satisfactory" and "poor", were first defined in the editor. To enable the system to calculate a similarity score during similarity retrieval, the symbolic values must be ordered. This could be done by assigning the priorities to the symbolic values in the symbol editor. In this example, "good" ranks higher than "satisfactory" which in turn has a higher rank than "poor".

Ideally, the ordered symbols should be modelled in Q-model within the performance module. However, since the value for the overall performance for each project must be transfered to the overall suitability and final scoring module for further assessment, two serious problems appeared. First, ordered symbols could not be transfered from one case-base to another. Secondly, when a qualitative model was developed, one or more virtual Q-nodes (can be thought of as a new intermediate field for holding the averaged total values of several fields) would be created. Also, the virtual Q-nodes created in the performance module could not be transfered to another module. These problems were due to the limitations of ReMind(tm).

To enable the performance score to be transfered to the overall suitability and final scoring module, the symbolic values in the performance module had to be left unordered. Rank ordering was assigned in the overall suitability and final scoring module by creating the same symbolic values as used in the performance module. Unordered symbols were transfered from the performance module to the overall suitability and final scoring module. These symbols were transformed into text through the text functions provided by the formula editor. The text value allows the system to find an existing ordered symbol that matches it and converts it back to a symbolic value for assessment. The ordered symbolic values transfered from the performance module were then modelled by the Q-model in the overall suitability and final scoring module (see Figure 9).

Figure 9 Modelling contractors' performance in the Q-model of ReMind(tm)

The rectangular boxes at the left-hand side of the Figure were the sub-criteria used for depicting contractors' financial performance. Information from three recent project reports would be used for this assessment. The financial performance for each project was summarised (see the second boxes from the left of the Figure). They were then averaged to determine the average score for the financial performance of the contractors (see the third box from the left of the Figure). This score together with the averaged scores from other performance-related criteria would become the score for the overall performance of the contractors (the right box in the Figure).

7.2 Finance Module

The finance module is responsible for storing the financial data of the contractors and converting the raw data into useful financial ratios for use in other modules. This module is perhaps the least complicated one in terms of case representation. Almost all data fields, except the name of the

contractor, the year of account and a confirmation for audited account, were numerical in nature. In ReMind(tm), a similarity score is automatically generated in each numerical field, such as contractors' turnover, according to the mean and standard deviation of the stored values. Therefore, no further models had to be developed to manipulate the information as would be required for symbolic fields.

Despite this, the similarity scores for most of the raw data, such as total asset or total current liabilities, are meaningless to the assessment. This data must be transformed into relevant financial ratios before a similarity score is computed. In this module, formulae of the required financial ratios were represented in the formula editor. The numerical values derived by these formulae would, in turn, be used to generate similarity scores.

7.3 System Input and Output

The system input and output provide the users with an interface for interrogating the system. System input included various on-screen forms for the entry of new cases and the maintenance of the existing cases. These forms were developed using the form editor of ReMind(tm). The system input forms were arranged into sections and the case features were grouped under the appropriate sections. To improve the legibility and user-friendliness, the features were converted to a question-answer format. Prompts were added to the system input forms to assist the users in answering the appropriate questions.

The output of the system is essential for decision support. Although the current prototype of EQUAL cannot produce structured reports, outputs significant to decision making are available. In the criteria formulation module, a report regarding the recommended decision criteria for the specified type of client and project is provided to the user. This report enables the client to prepare the prequalification questionnaire based on the recommended criteria. In the screening and reviewing module, the recommendation as to whether a contractor should be included in the next assessment process and the reasons for rejection or non-inclusion are reported. This information is useful to the client and contractors as it can be used to justify the decisions. The output of the overall suitability and final scoring module is the recommendation for the approved category and size of work a contractor should be included on a standing list after the overall suitability assessment. On the other hand, the names and scores of the best matching contractors generated by this module can be used for compiling the required list of tenderers. The output generated by EQUAL should assist the decision makers in performing the prequalification tasks more effectively and efficiently.

8. VALIDATION OF EQUAL

Validation determines if the right system was built. The validation of EQUAL took two forms: a Turing test and a face validation. A Turing test validates a system by asking the independent domain experts to evaluate the results provided by experts and results from the designed system without knowing the performers' identity (O'Keefe et al, 1987; Green and Keyes, 1987). This can eliminate any prejudice, for or against, using a computer system. A group of semi-experts were selected to take part in this test. Each of them was presented with two scenarios: one a civil engineering job and the other a maintenance job. The same scenarios were entered to the criteria formulation module of EQUAL. The outputs of EQUAL were manually transfered to the

answering sheet as used by the semi-experts. Two independent experts experienced in contractor prequalification were invited to assess the outputs. The results indicated that the solutions generated by the current prototype of EQUAL were generally more accurate than the semi-experts.

A face validation (O'Keefe et al, 1987) was carried out to identify the performance of EQUAL. During the face validation, a series of short demonstrations was undertaken by the author, and the domain experts were asked to use their knowledge and intuition to subjectively compare the system's performance against that of human experts (Anick, 1993). The participants of face validation included the experts who were or were not involved in the knowledge acquisition process. The experts in the face validation indicated that CBR was a suitable approach for modelling the prequalification tasks. The performance of EQUAL was highly satisfactory to the experts involved in the face validation.

9. CONCLUSIONS

A novel conceptual framework was proposed by the authors for the development of a specification for computer-based system for contractor prequalification. The framework consisted of system input and output and five interrelated modules: the criteria formulation module, screening and reviewing module, overall suitability and final assessment module, finance module, and performance module.

A new form of computer-based DSS for contractor prequalification was developed using the CBR approach. The prototype system - EQUAL - was developed, according to the established conceptual framework, using a PC-based CBR tool called ReMind(tm). The system begins by prompting the users to enter information regarding the client and project. It then searches its case-base for a similar case; that is a prequalification system of another client and/or category of work. Cases which best match the input information are retrieved, and if necessary, adapted to reflect the present situation. New solution, in terms of decision criteria for each assessment process, is generated by EQUAL for decision support.

EQUAL demonstrates that a practical solution can be produced even when the knowledge about a particular prequalification system is weak. The solutions obtained from previous cases can be modified to meet the current situation through the adaptation functions provided in the system. Both subjective and objective information can be manipulated in EQUAL to provide decision support. The prototype decision system tests indicate that the requirements of the prequalification domain can be addressed through the development of a CBR system.

ACKNOWLEDGMENT

This research is funded by the Engineering and Physical Science Research Council (EPSRC) under the grant number GR/J16121. The authors would like to thank UMIST, University of Leeds, Queensland University of Technology, Hong Kong Polytechnic University and all industrial collaborators for providing the requisite supports and information for this research.

REFERENCES

Aamodt, A., Knowledge-Intensive Case-Based Reasoning and Sustained Learning, Proceedings: ECAI-90, Ninth European Conference on Artificial Intelligence, August 6-10, Stockholm, Sweden, L.C. Aiello (ed.), Pitman Publishing, 1990, pp. 1-6.

Aamodt, A., Plaza, E., Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches, AI Communications, Vol. 7, No. 1, 1994, pp. 39-59.

Anick, P.G., Integrating Natural Language Processing and Information Retrieval in a Troubleshooting Help Desk, IEEE Expert: Intelligent Systems and Their Applications, Vol. 8, No. 6, December, 1993, pp. 9-17.

Bailey, S.F., Smith, F.C., Case-Based Preliminary Building Design, Journal of Computing in CivilEngineering, ASCE, Vol. 8, No. 4, October, 1994, pp. 454-467.

Barletta, R., An Introduction to Case-based Reasoning, AI Expert, August, 1991, pp. 43-49.

Brown, S.J., Lewis, L.M., A Case-Based Reasoning Solution to the Problem of Redundant Resolutions of Nonconformances in Large-Scale Manufacturing, In Innovation Applications of Artificial Intelligence 3, R. Smith & C. Scott (eds.), AAAI Press, 1993, pp. 121-133.

Department of Environment, CMIS, Contractor Management Information System, TheDepartment of Environment, 1992.

Green, C.J.R., Keyes, M.M., Verification and Validation of Expert Systems, Proceedings: WESTEX87, Western Conference on Expert Systems, Institute of Electrical and Electronics Engineers, 1987, pp. 38-43.

Kass, A., Adaptation-Based Explanation: Explanations as Cases, Proceedings: Sixth National Workshop on Machine Learning, June 26-27, Cornell University, Ithace, New York, Morgan Kaufmann Publisher Inc., 1989, pp. 49-51.

Kolodner, J., Extending Problem Solver Capabilities Through Case-Based Inference, Proceedings: Fourth International Workshop on Machine Learning, University of California, Irvine, June 22-25, P. Langley (ed.), Morgan Kaufmann Publishers Inc., 1987, pp. 167-178..

Kolodner, J., Case-Based Reasoning, Morgan Kaufmann Publishers Inc., 1993.

Latham, M., Constructing the Team: Final Report of the Government/Industry Review of theProcurement and Contractual Arrangements in the UK Construction Industry, HMSO, July, 1994.

Maher, M.L., Balachandran, M.B., Zhang, D.M., Case-Based Reasoning in Design, LawrenceErlbaum Associates, Inc., 1995.

Merna, A., Smith, N.J., Project Procured by Privately Financed Concession Contract, UMIST,1994.

Miller, P.L., A Critiquing Approach to Expert Computer Advice: ATTENDING, Research Notes in Artificial Intelligence 1, Pitman Advanced Publishing Program, 1984.

Ng, S.T., Skitmore, R.M., CPDSS: Decision Support System for Contractor Prequalification, CivilEngineering System, Vol. 12, 1995, pp 133-159.

Ng, S.T., Smith, N.J., Skitmore, R.M., Case-Based Reasoning for Contractor Prequalification - A Feasibility Study, Developments in Artificial Intelligence for Civil and Structural Engineering (ed. B.H.V. Topping), Civil-Comp Press, 1995, pp. 61-66.

Ng, S.T., Case-Based Reasoning Decision Support for Contractor Prequalification, A Thesis Submitted to the University of Manchester Institute of Science and Technology for the Degree of Doctor of Philosophy, April, 1996.

Ng, S.T., Skitmore, R.M., Smith, N.J., The Viability of Rationalising the Decision Criteria forContractor Prequalification, Construction Management and Economics, 1997a, In Press.

Ng, S.T., Smith, N.J., The Subjectivity of Contractor's Information and Assessment Methods forContractor Prequalification, Construction Management and Economics, 1997b, In Press.

Nguyen, V.U., Tender Evaluation by Fuzzy Sets, Journal of Construction Engineering andManagement, ASCE, Vol. 111, No. 3, 1985, pp. 231-243.

O'Keefe, R.M., Balci, O., Smith, E.P., Validating Expert System Performance, IEEE Expert, Winter, 1987, pp. 81-87.

Riesbeck, C.K., Schank, R.C., Inside Case-Based Reasoning, Lawrence Erlbaum AssociatesPublishers, 1989.

Russell, J.S., Skibniewski, M.J., Decision Criteria in Contractor Prequalification, Journal ofManagement in Engineering, ASCE, Vol. 4, No. 2, 1988, pp. 148-164.

Russell, J.S., Skibniewski, M.J., Qualifier-2: Knowledge-based System for Contractor Prequalification, Journal of Construction Engineering and Management, ASCE, Vol. 116, No. 1, March, 1990, pp. 157-171.

Schank, R.C., Kass, A., Riesbeck, C.K., Inside Case-Based Explanation, Lawrence ErlbaumAssociates, 1994.

Shimazu, H., Kitano, H., Shibata, A., Retrieving Cases from Relational Data-Bases: Another Stride Towards Corporate-Wide Case-Base Systems, Proceedings: Thirteenth International Joint Conference on Artificial Intelligent, August 28 - September 3, Chambery, France, Morgan Kaufmann Publishers Inc., 1993, pp. 909-914.