Optimal strategic recognition of objects based on candidate discriminating graph with coordinated sensors

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 4, JULYIAUGUST 1992 647

Optimal Strategic Recognition of Objects Based on Candidate Discriminating

Graph with Coordinated Sensors Y . C. Tang and C. S. George Lee, Senior Member, IEEE

Ahtract- Recognizing the objects using multiple sensors greatly relies on an appropriate sensing strategy that coordinates the sensors in a feasible and least expensive way to extract the information that is critical for recognition. An approach to this strategic recognition of objects based on the proposed candidate discriminating graph (CADIG) is presented. A CADIG represents the candidates of an unknown object and the complete discriminating relations between the candidates in terms of relevant discriminators (geometric features). Based on a CADIG, the strategic recognition is decomposed into two interacting procedures, the information acknowledgment procedure (IAP) and the sensory acquisition procedure (SAP). The IAP identifies the critical information for recognition as a set of discriminators from the CADIG and receives sensory examinations of these discriminators to eliminate invalid candidates from the CADIG. The SAP coordinates multiple sensors to examine the critical information identified in the IAP. Based on a proposed constraint-based dynamic model of sensors, this coordination is formulated as a constraint satisfaction problem and solved by the backtracking algorithm. The unknown object is recognized through iterations of the IAP and SAP until there is only one candidate left in the CADIG. The IAP and SAP are further integrated to achieve the optimal strategic recognition of objects. The amount of computations needed in this CADIG-based recognition is substantially reduced by pruning redundant discriminators from the CADIG. Reliable and knowledge-based recognition of objects is obtained by applying the Dempster-Shafer theory of belief functions. Computer simulations were performed to verify the feasibility and to analyze the performance of the optimal strategic recognition of objects.

1. INTRODUCTION ECOGNITION OF objects has increasingly and integrat- R edly used a variety of sensors [I]-[6]. The utilization

of multiple versatile sensors enables machines to recognize objects at a higher level of intelligence with more accuracy and reliability [7]. One substantial issue in achieving this performance is the exploration of sensing strategies that conduct adequate operations of multiple sensors for recognition. Such a sensing strategy needs to meet two requirements. First, it must be able to identify and purposefully extract the information of an unknown object that is critical for the recognition. As a result, the number of sensing operations required and thus

Manuscript received October 6, 1990; revised November I . 1991. This work was supported by a grant from the Ford Fund.

Y . C. Tang was with ATBT Bell Laboratories, Indian Hill South, Naperville, IN 60566-7045 and is now with ATBT Bell Laboratories, Holmdel, NJ 08827.

C.S.G. Lee is with the School of Engineering, Purdue University, West Lafayette, IN 47907-1285.

IEEE Log Number 9106515.

the cost of the recognition will be reduced. Second, in order to exhibit the integrated and intelligent behavior of multiple sensors, a proper coordination of sensors must be made in the sensing strategy to take the most advantages of the versatile capabilities of the sensors to extract the identified critical information in a feasible and least expensive way. The lack of such sensing strategies usually results in slower and more expensive recognition of objects. We refer to the recognition of objects conducted according to a proper sensing strategy as strategic recognition of objects.

Sensing strategies were seldom considered until in recent research by Grimson [8]-(91 and Hutchinson et a1 [lo]. Their approaches were successful, in some respects, to identify critical information of objects for recognition. The former explored the adequate positions of a tactile sensor at which crucial features can be extracted to disambiguate a set of candidates. This work, however, did not consider the employ- ment of multiple sensors for recognition. The latter proposed an optimal sensing strategy for recognition using multiple sensors. Given a set of sensors and their tessellated viewing points, the optimal sensing strategy at each iteration assigns a sensor to observe at a tessellated viewing point of the sensor such that the maximum number of candidates possibly remained for the unknown object after this sensing operation is minimized. The costs and practical restrictions for sensors to reach their viewing points were not considered in this optimal sensing strategy. Furthermore, this approach used only one of the multiple sensors at a time; the combined effect for recognition by using multiple sensors simultaneously as well as the coordination of sensors for doing so were not analyzed.

Research related to coordination of sensors for different tasks was reported in [11]-[13]. Beni et al. [12] provided an analysis of the dynamics of sensing operations. A team model was presented by Durrant-Whyte [I31 for the coordination of sensors in multisensor fusion tasks. The constrained spatial movement of a sensor for a particular task requirement was studied by Cowan and Kovesi [ll] in his optimal placement of a camera. Although their work contained a detailed analysis of the dynamics for a single sensor, it was not readily applicable to the context of multiple sensors.

This paper presents an approach for optimal strategic recognition of objects. This approach accomplishes model-based recognition of three-dimensional (3-D) objects. It generally allows simultaneous use of multiple sensors for recognition and therefore yields more efficient processes for recognition.

0018-9472/92$03.00 0 1992 IEEE

~

648 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS. VOL. 22, NO. 4, JULYIAUGUST 1992

Pre-stage - _ _ - Recognifi_on _.

Information cknowledgemen

Acquisition Procedure

I Recognized

Unknown Object

Fig. 1. Block diagram of the strategic recognition of objects with coordinated sensors

A block diagram of this approach is given in Fig. 1. Initially, a number of sensory observations of geometric features are arbitrarily taken on the unknown object and an initial set of candidates is obtained by comparing these observations with the relational models [14] of the known objects in the database using graph matching approaches [ 151 - [17]. Each candidate specifies the identity of the unknown object as well as its position and orientation (i.e., pose). In order to completely recognize the unknown object, further information useful for deciding the subsequent sensing operations is generated from graph matching. This is comprised in the candidate discriminating graph (CADIG) introduced in Section 11. A CADIG explicitly represents the candidates of the unknown object and the complete discriminating relations among these candidates in terms of relevant discriminators (geometric features). The characteristics and optimal pruning of the CADIG are investigated. Based on the CADIG, strategic recognition of objects is decomposed into two interacting procedures, the information acknowledgment procedure (IAP) and the sensory acquisition procedure (SAP). The IAP identifies the critical information for recognition from the CADIG as a set of discriminators and receives sensory observations of the discriminators to eliminate invalid candidates from the CADIG. Knowledge-based identification of critical information and reliable elimination of candidates in the presence of sensory noises are obtained using the Dempster-Shafer (D- S)’s belief functions [18]-[20]. The study of IAP is given in Section 111. The SAP, on the other hand, coordinates sensors to extract the critical information identified in the IAP. A constraint-based dynamic model of sensors is proposed to formulate this coordination as a constraint satisfaction problem, which is then solved by the backtracking algorithm. This is described in Section 111. The IAP and SAP form a closed loop and the unknown object is recognized through

iterations of these two procedures until there is only one candidate left in the CADIG. By considering the speed and the total sensing cost of recognition, the IAP and SAP are integrated to obtain the optimal strategic recognition of objects. This is discussed in Section V. Finally, computer simulations are conducted in Section VI to verify the feasibility and analyze the performance of the optimal strategic recognition of objects.

11. CANDIDATE DISCRIMINATING GRAPH (CADIG)

A. Discriminators and CADIG

A discriminator is a pair defined by d = ( f . p ) where f is the description of a geometric feature and p specifies the pose vector of the feature. The discriminator d is a supporting discriminator of a candidate if the feature f is present in the pose j-j on the candidate; otherwise, the discriminator is a rejecting discriminator of the candidate. Let us define &(c) and & ( e ) to be the set of supporting discriminators and the set of rejecting discriminators of a candidate e, respectively. Then, for a set of candidates C = { q. c2.. . . . c n } , the following relation holds:

A? (ez) = U” ,=lAs(c,)-As(cI) . / = 1 . 2 ;.., ri. (1)

Naturally, each pair of candidates, c l and c,(/ # J ) , in C can be distinguished by a discriminator d, which is not the supporting (or rejecting) discriminator of both c, and c J ; i.e., d satisfies

d E b.,(el) A d E &.(rJ) or d E b.,(r,) A d E a, ( c , ) (2)

where A represents the logical AND. For example, Fig. 2 shows two candidates c1 (dashed) and c2 (solid) of an unknown object. Let f l denote a corner feature and p l and p 2

TANG AND LEE: OPTIMAL STRATEGIC RECOGNITION OF OBJECTS 649

Fig. 2. Examples of supporting and rejecting discriminators of two candidates.

be two different pose vectors of the feature f l . Then d l = ( f l . p l ) and d2 = ( f l . p 2 ) form two discriminators, where d l E &(cl) A d l E A,(c2) and d2 E &(cl) A d2 E As(c2). According to (2), d l but not d2 can be used to establish the discriminating relation between c1 and c2. In general, based on the discriminators, the discriminating relations among a set of candidates can be represented as follows.

Definition 1: A candidate discriminating graph (CADIG) is defined as a four tuple Q = (C. A. 0. S d ) , in which G = (C. A ) is a directed graph with the set of nodes C and the set of arcs A where each node c, E C represents a candidate and each arc at, E A from node c, to cJ represents the discriminating relation between the candidates c, and c J ; 0 is the set of supporting discriminators of the candidates in C ; and &d : A + 2' is a mapping such that each arc c l iJ E A from candidate c; to candidate c, is attributed with a subset of 0, given by q 5 d ( a L J ) As(c,) - & ( c l ) . For each arc aiJ from candidate c, to c J , e, (c,) is called the active (inactive) candidate of a;,, or the active (inactive) candidate of each discriminator dl E $ d ( a L , ) .

From (l), it can be proved that each @ < , ( n l j ) in a CADIG is either empty or contains discriminators that satisfy (2); it therefore provides a valid discriminating relation between candidates c, and c,. An example of a CADIG is given in Fig. 3. Fig. 3(a) shows the candidates of an unknown object and their supporting discriminators denoted by arrows (considering only vertices and circular edges as features). Fig. 3(b) shows the CADIG generated from Fig. 3(a). A CADIG is complete if each pair of candidates in the CADIG is connected by at least one arc that is attributed with a nonempty set of discriminators. A complete CADIG provides the complete discriminating relations among its candidates. The CADIG in Fig. 3(b) is complete.

The validities of the candidates in a CADIG can be justified by sensory examinations of the discriminators in the CADIG. A discriminator d = (f .11) is said to be confirmed by sensors if the feature f is observed in the pose p on the unknown object. In such case, active candidates of d are verified to be valid whereas inactive ones of d are invalid and must be deleted together with their adjacent arcs from the CADIG. On the contrary, if d is not confirmed by sensors, active candidates of d are eliminated and inactive ones of d remained in the CADIG. A CADIG can be disambiguated in this way by successive sensory examinations of the discriminators. This disambiguation of a CADIG terminates normally when there is only one candidate left in the CADIG. In the following discussion, we shall prove that this normal termination is guaranteed for a complete CADIG, while for an incomplete CADIG there may be more than one candidate

I 3

9

( b)

Fig. 3. An example of CADIG. (a) The candidates and their supporting discriminators. (b) The CADIG generated from (a).

left indistinguishable after certain sensory examinations of the discriminators.

B. Disambiguation of Complete CADIG

Let C be a set of candidates and d be a discriminator. Define two complementary subsets of C as Gd = {clc E G and d E A,(c)} and C; = C - Cd. We say that C can be divided by d into Cci and C;i if Cd and Gx are both nonempty. Based on these notations, the disambiguation of a CADIG based on sensory examinations of the discriminators can be represented by a binary tree in the following definition.

Definition 2: A candidate dividing tree (CDT) is a binary tree T defined by a set of candidates C and a sequence of discriminators in D such that T is expanded by the sequence of discriminators in D from the root n o associated with Co = C in the following ways: 1) the expansion of T by a discriminator tl is obtained by expanding each leaf node of T by d if it is expandable, and 2) each leaf node nt associated with Cz is expandable by a discriminator d into two descendant nodes if C' can be divided by d into two (nonempty) subsets CA and C;. In the expandable case, the two descendant nodes are associated with Ci and .

The CDT defined above contains all the possible results (remaining valid candidates) from the disambiguation of C by D. Each possible result can be given by tracking a path from the root to a leaf node on the CDT according to a particular result of the sensory examinations of D-the remaining valid candidates are specified by the set of candidates associated with the leaf node. An example of CDT is given in Fig. 4, representing the disambiguation of the CADIG in Fig. 3(b) by the sequence of discriminators d 1 , dy. and dq. A CDT is complete if and only if each leaf node in the CDT is associated with a set of single candidate. The CDT in Fig. 4, for instance, is complete. Obviously, a complete CDT generated by G and D indicates that the unknown object can always be recognized from C based on sensory examinations of D. We next prove

650 LEE€ TRANSACTIONS O N SYSTEMS, MAN, AND CYBERNETICS. VOL. 22, NO. 4, JULY/AUGUST 1992

Fig. 4. An example of CDI

that a complete CDT is a necessary and sufficient condition of a complete CADIG.

Theorem 1: Let 9 be a CADIG with the set of candidates c and the set of discriminators 0. Then P is complete if and only if the CDT defined by C and 0 in any sequence is complete.

Proofi 1) Suppose \zI is complete. Then for any two candidates in 9, there must be an arc of discriminating relation in between which is attributed with a nonempty set of discriminators. Based on any sensory outcomes of the discriminators in 0, at least one of the two candidates must be eliminated from 9. Therefore, it is impossible to have more than one candidates left indistinguishable after the sensory examinations of 0 and the CDT defined by C and (3 must be complete. 2) Suppose the CDT defined by (? and e) is complete. Assume 9 is not complete. Then there exist two candidates c, and cJ in P, which are not discriminated by any discriminator in 0; that is, ( , O d ( a l , ) = <) ( ! (aJ , ) = a. Since q5d(aij) = & ( c l ) - a,s(ci) , the supporting discriminators of c, and c j must be identical. Hence, there exist certain sensory outcomes of 0 that would confirm every supporting discriminator of both c; and cj and leave the two candidates indistinguishable. This contradicts our premise that the CDT

U From Theorem 1, it is a direct consequence that the dis-

ambiguation of a complete CADIG is guaranteed to terminate normally with only one candidate left for the unknown object.

is complete. Therefore, P must be complete.

C. ODtimal Pruning of Discriminators in CADIG - "

The disambiguation of a CADIG, as will be shown later, requires searching for the discriminators that provide the most critical information for the recognition. The order of the complexity of this search is essentially exponential in the number of the discriminators in the CADIG. For a CADIG containing numerous discriminators, the amount of computations needed is prohibitively large. Fortunately, for most cases, the discriminators in a CADIG are redundant in representing the discriminating relations. These discriminators can be pruned without affecting the result of recognition by eliminating all but a subset of the discriminators that keeps the pruned CADIG complete. Since each discriminator usually has a cost for the sensors to examine it, the optimal pruning of discriminators is achieved by retaining a completeness-preserving subset of

for a path from the initial state to a goal state of the database. This state-space searching problem has been known to be solvable by the A* algorithm [21], [22] based on certain heuristic function. This algorithm also guarantees a minimum- cost-path solution if a solution exists. By assigning the cost of each link expanded on the search graph by the sensing cost of the discriminator, the A* algorithm thus provides a solution for the optimal discriminator pruning of the CADIG. Detailed steps of this solution are given in the il*-based algorithm optimal discriminator pruning (ODP) in the Appendix.

Given a CADIG, algorithm ODP determines the completeness-preserving subset of discriminators, D*, from the CADIG with the minimum-total-sensing cost. In algorithm ODP, the evaluation function f ( r i ) computed at each node n for selecting the most promising node for expansion is defined by

f ( r 1 , ) = g(71) + h ( 1 1 ) ( 3 )

where g ( n ) is the cost of the minimum-cost path from node 77, to the starting node found so far recorded by pointers on the search graph, and h ( n ~ ) is an estimate of the cost of the minimum-cost path h * ( n ) from node 71 to a goal node. For an A* algorithm, it is required that the heuristic function h ( 7 1 ) be properly defined as an underestimate of h*(n) [21], [22]; that is, h ( n ) 5 h * ( r t , ) . In algorithm ODP, the heuristic function is given by

(4)

where c p c ( d ) is the sensing cost of a discriminator d, and D,, is the set of discriminators that can expand node 71 (see the Appendix). It is obvious that the minimum-cost path from node 71 to a goal node must consist of one of the links expanded from node 71 and since the cost of a link is given by the sensing cost of the discriminator that expands this link, an underestimate (lower bound) of / & * ( / a ) can be given by (4). Therefore, I L ( ' ~ L ) in (4) is a valid heuristic function for the A* algorithm ODP.

Fig. S shows an example of the optimal discriminator pruning for the CADIG in Fig. 3(b) based on algorithm ODP and the sensing costs of discriminators listed in Table I. The expansions of the nodes in the A* search are illustrated on the search graph in Fig 5421) by dashed lines, and squared numbers on the nodes show the order of expansions. The global database at each node is characterized by the sets of the candidates on the leaf nodes of the corresponding CDT. The minimum-cost path from each expanded node to the starting node is directed by pointers. Eventually, the A* search terminates with D* = { d l . d j . d4}. The pruned CADIG is then given in Fig. S(b). In this example, the number of discriminators is reduced from 10 to 3, which greatly reduces the amount of computations needed in the later procedures.

discriminators that requires the minimal total sensing cost. According to Theorem 1, a completeness-preserving subset 'I1. INFoRMATIoN

of discriminators in a CADIG corresponds to a sequence of discriminators that expands an initial CDT to a complete one. Given that the global database be the expanding CDT and a goal state of the database be a complete CDT, the search for completeness-preserving discriminators amounts to the search

Based on a CADIG, the information acknowledgment procedure (IAP) in Fig. 1 concerns two major tasks: disambiguating the candidates based on sensory examinations of the discriminators and selecting the discriminators to be examined in the next sensing operations. This section shows how to

TANG AND LEE: OPTIMAL STRATEGIC RECOGNITION OF OBJECTS 65 1

( b)

Fig. 5. Optimal discriminator pruning of the CADIG in Fig. 3(b) according to algorithm ODP. (a) The .A* search graph. (b) The pruned CADIG.

TABLE I A LIST OF SENSING COST5 FOR THE DISCRIMINATORS LN THE CADlG OF FIG. 3( b).

Discriminator dl dz d.3 d l (1; d: (lq cl<) r l io cost 3 1.5 1 3 1 1 1 1.5 1.5 1.5

achieve reliable disambiguation of candidates in the presence of sensory noises, and how to select the discriminators for sensory examinations that are likely to render the largest eliminations of invalid candidates from the CADIG based on some knowledge about the unknown object.

A. Reliable Disambiguation of Candidates

Fig. 6 shows the general discriminating relation between a pair of candidates c, and cg in a CADIG, where df ( I = 1 to 7 ) )

and d i ( k = 1 to m) are supporting discriminators of c, and r,, respectively. Let a; be a two-valued function defined as the outcome of the sensory examination of df such that a; = 1(-1) if 4 is (is not) confirmed by sensors. Similarly, ai is defined as the two-valued outcome function for d i . The disambiguation of a pair of candidates is quite simple in the noise-free situation since inconsistent outcomes (evidence) are prohibited from the sensory examinations.

In the presence of sensory noises, however, contradictory outcomes of sensory examinations may be present and must be carefully considered as evidence for the disambiguation of candidates. We shall achieve this by using the D-S theory of belief functions [ 181-[20] to represent uncertain and mutually exclusive evidence in the disambiguation of candidates. In this method, each outcome of sensory examinations forms a piece

a’

d; , a:,

Fig. 6. General discriminating relation between a pair of candidates

of evidence that is attributed with a value representing the D-S belief in that this evidence is true. This belief reflects the degree of the sensory noises and can be measured from the operational performance of the sensor. In disambiguating candidates, these belief-attributed pieces of evidence are integrated according to Dempster’s rule of combination [ 181 to make decisions of eliminating candidates. As a result, the effect of sensory noises is reduced and reliable disambiguation of candidates can be obtained.

This reliable disambiguation of candidates can be illustrated by a simplified example below. Suppose two candidates e1 and c2 in a CADIG are distinguished by the supporting discriminators d l of c1 and dz of c2 only (Fig. 7(a)), and a1

and a2 are the outcome functions for d l and dz, respectively.

652 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS. VOL. 22, NO. 4, JULYIAUGUST 1992

< - - - b2

ai = 1

Bel(C2)=bl + b2 - bibz =-1

(a)

I 4 1 - - al=l 0

a2 = 1 Bel(ClnC2) = b l bz Bel(C~)=bi(l-b2)

(b )

Fig. 7. Disambiguation of a pair of candidates based on two different sensory outcomes. (a) 0 1 = 1, o 2 = -1 . (b) ( 1 1 = 0 2 = 1.

Let a1 be attributed with b l that gives the belief in the value of a l , and similarly a2 be attributed with b2. Define the frame of discernment as two propositions PI and €‘?, where Pl asserts that ‘‘q is a valid candidate” and P2 asserts that “Q is a valid candidate.” Let A1 and A2 be the propositions that will be asserted based on the values of N I and 0 2 , respectively. Then we have

where P , denotes the complement of the proposition P,. Consequently, the beliefs in the propositions A I and A2 can be given by the basic probability assignments rn1 and m 2 ,

respectively.

where R is the universal proposition. In order to make a proper decision of eliminating candidates, it is necessary to combine ml and m2 according to the outcomes of sensory examinations. To show this, let us look at the instance of sensory outcomes given by a1 = 1 and (12 = -1. From (S), we have A1 = A2 = p2. The combination of 1/11 and rri2

in (6) using Dempster’s rule of combination then yields the belief function

bel(P2) = bib2 + b l ( l - b 2 )

bel(R) = 1 - hel(P2)

where bcl(R) is the fraction of uncertainty in the combined belief function (Fig. 7(b)). If the fraction of uncertainty is comparatively small such that it can be implied that the belief in the proposition P2 is convincing enough, it is proper to eliminate the candidate c2 rather than c1 from the CADIG. If the fraction of uncertainty is too large, no candidate should be eliminated since the evidence from sensory examinations is too uncertain to make any decision of elimination.

Fig. 7(c) shows another instance of sensory outcomes with a1 = cy2 = 1. In this case, A1 = P2 and A2 = PI.

-

Accordingly, the combined belief function is given by

bel(P2) = b l ( 1 - b 2 )

bel(P1) = b*(1 - b l )

be l (P1 nP2) = blb2

bel(R) = 1 - hel (P1) - bel(P2) - bel(P1 n P z ) . _ _

We now have beliefs in three propositions P2, P I , and PInP, regarding eliminating candidate c1, candidate c2, and both candidates c1 and c2, respectively. The choice of elimination supported by the maximal belief is then taken provided that the fraction of uncertainty bel( 0) is comparatively small. For other instances of sensory outcomes, the appropriate choices of elimination can be taken in a similar way.

The disambiguation of a pair of candidates with the general discriminating relation in Fig. 6 can be extended from the fore- going results. Let the outcome function cy; for the discriminator d ; be attributed with the belief bj for 1 = 1 to m, and the outcome function 0; for d i be attributed with the belief b i for k = 1 to 71. Define PI as the proposition asserting “cl is a valid candidate.” On combining all the beliefs, we have the following results for decision-making:

h?l(P,) = P,/(l - K ) bel(F,) = BZ/(1 - K ) bel(17, np,) = BZ,/(i - K ) hd(12) = Bo/(l - K )

(7) (8) (9)

(10)

where K = 1 - B, - r, - Bz, - Bo is the factor of contradiction and

7 r i ri

(14) 1=1 I=1

In (11) and (12), qJ ( q z ) is a constant of value 0 if there is no direct evidence for the invalidity of candidate cl (cz); that is. when a; = -1 for all 1 and U ; = 1 for all IC (when af = 1 for all 1 and ai = -1 for all I C ) . Otherwise, ql ( q z ) has the redundant value of 1. Similarly, in (131, qtJ is a constant of value 0 if af = -1 for all 1 or ai = -1 for all I C , and of value 1 otherwise.

Let T be a threshold giving the maximum fraction of uncertainty allowed in (10) such that the beliefs in (7)-(9) would be convincingly enough. Consequently, the disambiguation of candidates c, and cJ is conducted by (1) eliminating no candidate if bel(R) > T , and (2) otherwise, eliminating


candidate cJ if bcl(pJ) is maximal, eliminate candidate e, if bel(P,) is maximal, and eliminating both candidates c, and cJ if b e ~ ( P , nP,) is maximal.

B. Effective Discriminators for Recognition

The performance of CADIG-based object recognition relies on the number of eliminations (ne ) achieved at each iteration of the recognition. A large value of 71, at each iteration gives rise to fast and frequently less expensive recognition of objects. Let D be a set of discriminators. Due to the indetermination of sensory results, the value of n, rendered by the sensory examinations of D is random. We therefore define the expectation of TI,, as the entropy of D , denoted by H ( D ) , serving as a measure of the effectiveness of the information carried by D for the recognition. Some previous work [ lo] has suggested the minimal possible value of TI , as the entropy. Based on the measure of entropy, at each iteration the set of discriminators with the maximum entropy is selected for the next sensory observations.

The possible values of n , rendered by a set of discriminators can be exhausted on the corresponding CADIG. Without further information, the probability distribution of these values is assumed to be uniform. Very often, however, one may have extra knowledge about the unknown object that is useful for its recognition and would provide information for the probability distribution of 71 , . For instance, one may collect a strong belief in that a particular geometric feature is present on the unknown object from previous results of recognition. This belief would indirectly contribute a greater probability to certain values of 7 1 , and thus refine the assumed uniform distribution of 1 1 , for obtaining better computations of entropy.

An example of this knowledge-based computation of entropy is illustrated in Fig. 8, where the entropy of { tll} for the pruned CADIG of Fig. 5 is computed. The possible values of n, rendered by { d l } are 1 and 3; they are accumulated with equal probability mass as shown in Fig. 8(a). Without further information, the entropy is given by H ( ( d 1 ) ) = 2. Now suppose we have two sources of recognition knowledge, K 1 and K2, regarding the likely presence of certain discriminators on the unknown object. This knowledge, expressed in D-S’s belief functions, is given by the basic probability assignments

for source K1 and

for source Kz, where a, is the outcome function for the discriminator d,, i = 1. 3, 4. On combining the two knowledge sources, we have the basic probability assignment 7 r t ~ = mK1 @ m K 2 given by

~ n, Bel(ne=3)=0.7 Bel (n,=l, 3H.3 W ( d i 1 I 0 = 2.7

( b)

Fig. 8. Computations of entropy for { ( r , } . (a) Without recognition knowledge. (b) With the recognition knowledge Ii in (15).

From (15a) and (I%), we have l)el(trl = 1) = 0.28 + 0.42 = 0.70. Since the event of “ T i f , = 3” can be deduced from ‘‘(L~ = l”, the belief in ‘‘n? = 3” is also 0.70. Consequently, i t can be partially clarified that at least 70% of the probability mass should be accumulated on “ T I , = 3”. Since the beliefs in (15) are unable to completely specify the allocation of probability mass, the remainder of the probability mass is still equally distributed to the events of “ n P = 1” and “n.? = 3.” The total probability of “71, = 3 ” then becomes 0.7 + 0.312 = 0.85 and that of ‘“nf, = 1” becomes 0.3/2 = 0.15. This refined probability distribution of T I , , is shown in Fig. 8(b). Thus, the entropy of rll now computed with the recognition knowledge is given by H ( { d 1 } . K ) = 2.7.

This knowledge-based computation of entropy can be conducted in the following systematic procedure. Given a CADIG 9 = ((7. A . 0. dd), let D c 0 be a subset of discriminators and To be the CDT generated by C and D according to Definition 2. Note that the number of candidates associated with each leaf node of To indirectly provides possible values of n , rendered by D . Let the recognition knowledge be generally expressed by a combined D-S belief function that assigns beliefs to a set of propositions, each of which asserts particular sensory outcomes of D (for example, the recognition knowledge expressed by (15)). Then, the entropy of D can be derived as follows. For each belief-assigned proposition P in the recognition knowledge, a multi-path tour on TD from its root node is directed in the way that, at each node n7, which is associated with C, and expanded by a discriminator d, 1) the path from 7ai to its descendant node associated with C,, (or Cia) is traveled by the tour if the sensory outcome of d is specified to be 1 (or - 1) in the proposition P, and 2) both paths to the descendant nodes are traveled if the sensory outcome of (1 is not specified in P. Consequently, for each leaf node of TD thus visited by the multipath tour, its associated set of candidates provides one possible set of remaining candidates (and therefore one possible value of rr,) that can be deduced from the proposition P. The belief assigned to P is then evenly distributed to all the possible values of n, found on the multipath tour. The probability distribution of n, is thus refined by executing the above steps for all the belief-assigned propositions in the recognition knowledge.

According to the above procedure, the entropies of other discriminators in the pruned CADIG of Fig. 5 can be derived. They are computed based on the knowledge in (15) and are listed in Table 11. As a result, the set of discriminators { dl. d4} with the maximum entropy is selected for the next sensory examinations.

654 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 4, JULYIAUGUST 1992

TABLE II ENTROPIES OF DISCKIMINATORS COMPUTED WITHOUT AND wlnl THE KNOWLEDGE IN (15)

{ d l } { d 3 ) {d4} {dI.d,31 {dI .d41 td:3.&/

2.0 2.0 2.0 2.5 2.5 2.5 Entropy without Ii Entropy with Ii 2.7 1.54 2.0a 2.85 2.94 2.3

~~ ~ ~

aIn this case, B d ( I ! , = 2 ) = 1 0 and the knowledge

IV. SENSORY ACQUISITION PROCEDURE The sensory acquisition procedure (SAP) executes the ex-

aminations of the discriminators requested by the IAP by using various physical sensors. This section discusses how to accom- plish this in minimal total sensing cost by proper coordination of multiple sensors subject to the dynamics of the sensors.

A. A Constraint-Based Dynamics Model of Sensors

A dynamics model of multiple sensors for recognition is first proposed in a general sensing environment. In the environment, each sensor is represented by a module similar to a logical sensor [23]. Each module contains a physical sensing device and a processing unit for generating descriptions of geometric features from raw sensory data. Depending on the type of the sensing device and the capability of the processing unit, a limited set of geometric features can be measured by each sensor. Each physical sensing device has an operational range within which the set of measurable geometric features can be observed. For example, the operational range of a camera is specified by its field-of-view while a tactile sensor is restricted by the contact area of its sensing array. Each sensing device also has the flexibility of being located with certain degrees of freedom within a restricted working space when mounted on a moving carrier (e.g., a robot manipulator or a moving platform). In addition, it is assumed that for each sensor there always exists a minimum-relocating-cost path from its current pose to a desired pose within its working space that can be accomplished by its carrier. In summary, the set of measurable features, the operational range, and the working space are used to completely specify the capabilities of a sensor for recognition. In practice, these capabilities are limited. We thus present a dynamics model of sensors, based on by the following sensor assignment and allocation constraints (SAAC), to describe the restricted capabilities of sensors in examining discriminators for recognition.

1) Sensor Assignment Constraint: A sensor can examine a discriminator if the geometric feature specified by the discriminator is measurable by the sensor.

2) Sensor Allocation Constraint: A sensor can examine a set of discriminators if there exists a feasible pose of the sensor within its working space such that in this pose the sensor can examine within its operational range each of the discriminators simultaneously without occlusion.

For the sensor allocation constraint, the set of discriminators may be occluded by the unknown object itself. For instance, consider a discriminator in the set that is present on a particular candidate of the unknown object. This discriminator may not be observable by a visual sensor within its working space because the discriminator is occluded by the candidate itself. In

I\‘ has no effect

A check for multiple-to-one assignment constraint x satisfaction fails 0 a feasible instantiation of a

( b)

Fig. 9. Coordination of sensors formulated as a CSP. (a) The bipartite graph representation. (b) The search graph.

order to allow the discriminator to be examined by the sensor, there must exist one feasible pose of the sensor in which the discriminator can be examined without being occluded by the volumetric union of all the candidates of the unknown object that contain the discriminator. The feasible poses of the sensor for examining the set of discriminator without occlusion are therefore the intersection of the feasible ones obtained for each discriminator [ 241.

B. Coordination of Sensors as Constraint Satisfaction Problem

Based on the SAAC dynamics model of multiple sensors, the coordination of sensors in SAP constitutes a constraint satisfaction problem (CSP) [25]-[27]. Fig. 9(a) shows a bipartite graph representation of this CSP in which D = { d l . d2, . . . , d m } is a set of discriminators to be examined and S = {SI. ~ 2 , . . . . s p } is a set of sensors available for the examinations. Each edge on the bipartite graph illustrates one assignment of a discriminator to a sensor for examination. We constrain the assignments on the bipartite graph to be function- like; that is, each discriminator cannot be assigned to more than one sensors. This excludes the consideration of sensor fusion and simplifies our following discussion. Hence, all possible assignments of discriminators to sensors can be represented

TANG AND LEE OPTIMAL STRATEGIC RECOGNITION OF OBJECTS 655

by an m-tuple of variables a = ( a l . u2. . . . . n,,,), where each variable a; has a domain Si c S of possible values, and the value of the variable ai denotes the sensor to which the discriminator d, is assigned. Define A = S1 x 5'2 x . . . x S,,, as the assignment space. Then, a feasible coordination of sensors is described by an instantiation of a in A that satisfies the constraints of SAAC.

According to the SAAC, we first restrict the domain Si of each variable 0, to contain only the sensors in S, each of which is able to examine d , without occlusion in a feasible pose within its working space. This restriction of domains satisfies most of the SAAC except for the multiple-to-one assignment situation-each instantiation of a that assigns more than one discriminators to a single sensor must be checked to see whether those discriminators can be properly observed by the sensor simultaneously. The set of all feasible instantiations of a is therefore a subset of A that further satisfies the multiple- to-one assignment constraint. This constitutes the CSP to be solved in the following discussion.

The backtracking algorithm [25] is employed to solve our CSP. In doing so, the backtracking algorithm explores the assignment space A by sequentially instantiating the variable

from 'I = 1 to m on a search graph. For the instantiation of variable n,, the backtracking algorithm assigns a value to i t from its domain S,, and examines the consistency of the value with previous instantiations of variables whenever the value has been assigned previously to other variables. In such case, more than one discriminators have been assigned to a sensor and the multiple-to-one assignment constraint must be particularly examined. If the value does not satist'y the constraint, the backtracking algorithm goes back to instantiate ai by another value remaining unassigned in S , ; otherwise, the next variable fi,+1 is instantiated. Consequently, a feasible instantiation of a is derived whenever a path of length rr) is reached on the search graph, and all the feasible instanti.stions of a are obtained by applying the backtracking algorithm throughout the entire search graph.

Fig. 9(b) shows an example of the backtracking algorithm applied to a search graph with t n = -1 and p = 3. Node ,t~,o denotes the root of the search graph. Initially at node no. the variable a 1 with domain 5'1 = { S I . s'. .s: j} is instantiated by the value .sl and the node is expanded from tt,i) by a solid arc. The dashed arcs at n o provide the alternative expansion from 710 with other instantiations of (il. The subsequent expansion of solid arcs shows the instantiations of variables continued on the search graph. The satisfaction of the instantiated values are checked at nodes a4. I ! : , and 71,9,

since at these nodes more than one discriminators are assigned to the sensors SI. Failures of this check are indicated by an x at nodes 717 and 719; the descendant subgraphs of 717

and n~ are thus excluded from the search graph. A solution of the backtracking algorithm is exhibited by the path from node n o to node 7110 that provides a feasible instantiation

From practical and performance point of view, an additional constraint that restricts the maximal number of discriminators assigned to a sensor can be imposed in the CSP. This is because that most sensors with restricted sensing capabilities

(SI. SI. S A . S j ) .

are unable to provide fine sensory observations for a large set of discriminators distributed over a wide range of positions. The additional constraint then decreases the possibility of poor sensory observations of discriminators and thus im- proves the performance of recognition. Another advantageous consequence of this extra constraint is that the number of nodes expanded on the search graph and the time-consuming checking of the multiple-to-one assignment constraint will decrease. As a result, the amount of computations needed in the backtracking algorithm is significantly reduced.

v. INTEGRATION OF IAP AND SAP

So far, we have presented the strategic recognition of objects in two independent procedures: the set of discriminators with the maximum entropy is selected in the IAP, and they are examined by proper coordination of sensors in the SAP. As a result of independence, the IAP tends to select as many discriminators for sensory examinations as needed to maximally disambiguate the candidates in order to achieve fast recognition. This in terms yields a high total sensing cost in the SAP that may not be desirable. A tradeoff between the speed and the total sensing cost of recognition is therefore required. This can be achieved by integrating the IAP and SAP to perform the optimal strategic recognition of objects.

At each iteration of the recognition, let 0 be the set of discriminators in the current CADIG. The CSP formulated in the previous section is extended by first extending the set of discriminators considered to be assigned to sensors to the set (3, and a null sensor S O is added to the domain Si of each variable ai with the instantiation a, = S O meaning that the discriminator d, is not examined by any sensor. The solutions of the extended CSP, except the trivial one having ai = SO for all i , then provide all the feasible coordinations of sensors for the current iteration of recognition. Among these feasible choices, the one that optimizes a given global performance evaluation function is selected.

A global performance evaluation function should consider at least the two essential criteria: the speed and the total sensing cost of recognition. The speed of recognition can be measured by the entropy H * ( h ( a ) , K ) , where h(a) denotes the set of the discriminators assigned to nontrivial sensors (excluding s o ) in the assignment a, and K is the recognition knowledge if any. The total sensing cost of recognition can be given by two measures, .(a) and ~ ( a ) . .(a) is the minimum total cost (energy) required to relocate sensors from their current poses to certain ones for properly examining the discriminators assigned to them in a, and accordingly T ( U ) is the total cost (data processing time) for the sensors to examine the assigned discriminators. These two measures can be constructed by

In (16), 1: is the current pose of sensor si, 1: is the pose of sensor at s, at the end of its minimum-cost relocating path, and


Fig. 10. CAD model of known objects in computer simulations.

c*( I p. I :) denotes the cost of moving the sensor ,s, from ly to

1;. The computations of If and c, vary for different sensors and their carriers. For a sensor carried by a platform moving in a restricted two-dimensional (2-D) area, the computations of relocating costs could be linear and straightforward. For sophisticated carriers such as robot manipulators, however, these computations are complicated. In (17), 71, , ( a ) denotes the number of discriminators that are assigned to sensor s , in a and contain geometric features of type 1, and t,, is the data processing time at s, for examining a geometric feature of type J .

In general, a global performance evaluation function can be chosen in various forms for different applications. An illustrative example is given by

where d1 and w2 are two nonnegative weighting factors. According to (IS), the feasible coordination of sensors solved from the extended CSP and maximizing (18) is selected at the current iteration of recognition, and thus the optimal strategic recognition of objects is obtained.

VI. SIMULATION RESULTS

Computer simulation software with interactive graphic display was developed on a Sun 3 workstation to verify the performance of the proposed optimal strategic recognition of objects. This graphic system is designed with the ability of 1) interacting with the database of a CAD model to obtain an initial set of candidates, 2) defining the restricted capabilities of multiple sensors, and 3) creating and displaying simulations of the optimal strategic recognition of objects. The set of objects from which an unknown object is recognized is described by the CAD model in Fig. 10. Two visual sensors are employed


Fig. 11. Configurations of the sensors in computer simulations

for the recognition; each of them is capable of moving within the restricted planar working space outlined in Fig. 11 and measuring 3-D geometric features of circles and corners in the fixed viewing direction perpendicular to its working space with 45 degrees field-of-view angle. It is assumed that, from a few sensing operations, the initial set of candidates of the unknown object and their supporting discriminators d , , i = 1.2 , . . . . 18, are given in Fig. 12 (discriminators denoted by subscripts only). Sensory measurements of the unknown object are simulated by perturbing the CAD model of the unknown object by normally distributed random noises. Simulation parameters are chosen as follows. For both sensors, a linear sensor relocating cost is assumed in (16); that is, c; I o IT ki 1; - I F with k1 = k.2 = 1.0 in the simulation. The unit data processing time t ; j in (17) is set to be 1.5 for circles and 1.0 for corners at sensors 1, and 1.0 for circles and 0.75 for corners at sensor 2. The threshold of uncertainty, T , for reliable disambiguation of candidates is given as 0.75. Two global performance evaluation functions given in (19) and (20) are used in the simulations:

( t 4 = I I

In each simulation, the optimal strategic recognition begins with the CADIG generated from the candidates in Fig. 12

and optimally pruned by algorithm ODP to contain only the set of discriminators { d l , d3. &. (15. d g } . By choosing the unknown object to be candidate ~ 1 , Fig. 13 graphically displays the simulation of optical strategic recognition based on the global performance evaluation function f,' in (19) and no recognition knowledge. Table 111-A shows the numeric results of the simulation. The first row gives the initial lo- cations of the sensors. The successive rows then show the coordination of sensors selected at each iteration and its resultant entropy, data processing time, relocating cost, and remaining valid candidates. For example, at the first iteration, the coordination of sensors assigns the discriminator dg to sensor 2 that then moves from location (7.0,0.0,4.0) along the minimum-relocating-cost path to location (5.5,0.0,3.5) in order to properly examine d:.

With the same unknown object, the results of simulation based on the other global performance evaluation function f,' in (20) and the recognition knowledge

K3 : hrl(fi4 = 0) = bc'l(fis = 0) = 0.4

are provided in Table 111-B. It can be seen that the increased emphasis of f,' on the entropy now encourages the sensors to examine more discriminators at each iteration and hence results in faster recognition with a smaller number of iterations. This trend is further illustrated in Table IV, where a comparison is made between the performance of recognition obtained based on f: and f z , respectively.


Candidate 2 Candidate I

Candidate 4

Candidate 7 Candidate 8

Fig. 12. Initial set of candidates for the known object in computer simulations

Finally, the advantage of knowledge-based recognition VII. CONCLUSION

is Let the unknown Object be se- This paper presented the optimal strategic recognition of lected from cl to but with a high probability of being either c7 or c8. After a few simulations of recognition, we collect the recognition knowledge

objects based on the proposed CADIG. A CADIG is able to represent the complete discriminating relations between the candidates of an unknown object in terms of discriminators. The redundant discriminators in a CADIG were optimally pruned using the A* algorithm ODP and, as a result, the amount of computations needed in the CADIG-based recog-

K4 : bel(a4 = 1, a5 = 0) = 0.7.

Then, a comparison is made in Table V between the performance of recognition obtained with and without the recognition knowledge K4 (both based on f,"). It is shown that, for the recognition conducted based on K4, fewer discriminators need to be examined by sensors during a smaller number of iterations in order to completely recognize the unknown object. This can be explained by that the knowledge K4 is able to direct the recognition process to examine those discriminators that are likely to be critical for the recognition. This usually yields a better and intelligent performance of recognition.

nition was reduced. Two major results were obtained: the information of an unknown object critical for its recognition is identified, and multiple sensors are coordinated in a feasible and optimal way to extract the critical information. For the former, we showed the critical information for recognition can be identified as a set of discriminators from a CADIG based on knowledge-based measurements of entropy. Sensory examinations of these discriminators with noises were used to eliminate invalid candidates from the CADIG in a reliable decision-making procedure. These were accomplished in the IAP. For the latter, the coordination of sensors to examine

TANG AND LEE: OPTIMAL STRATEGIC RECOGNITION OF OBJECTS

Strateglc rccognltlon at 1teratl.n 1

se06.r used: s 2 f eotrrpy = 4 . 0 0 process tlnt = 0 .75 s e n s l n g cost = 2.31 caadldates left: c6/cl/c~/cS1cl1c21c~1

Stratelk rtcr(i1flei a t Iteratlm S

sensor used: r2/ entropy = 3.00 process tlnt = 0.15 senslng cost = 5 .00 cmdldates left: c8/cl/cS/

mug.: wt: .

\ ,'

Strategic recognition at Iteratloo 2

sensor used s21 entropy = 3.60

5ens10g cost = 0 . 7 , caod I dates I eft : c 7 l c 8 I c 5 ,' c 1 / c2 / c 3 I

p r o c e s s tine = l . o o c

Strateglc recognltlon at I t e n t i e n 4

sensor used' S l l entropy = 1 .50 process t l R = 1.50 sensing cost = 4 . 8 4 candidates left: c l l C 3 1

(c)

sensor used a l l entropy = 1.00 process tlne = 1.00 senslng cost = 4 .80 candldates left clf

:andid Sensor

osla Sa".

Print "it

Fig. 13. Graphic display of computer simulations. (a) Strategic recognition at iteration 1. (b) Strategic recognition at iteration 2. (c) Strategic recognition at iteration 3. (d) Strategic recognition at iteration 4. (e) Strategic recognition at iteration 5.

critical information was formulated, subject to the proposed SAAC-based dynamic model of sensors, as a CSP and solved by the backtracking algorithm. These were achieved in the SAP. Based on a global performance evaluation function, the IAP and SAP were integrated to obtain the optimal strategic

recognition of objects. Computer simulations were conducted to verify the feasibility and to analyze the performance of the optimal strategic recognition of objects.

Several aspects of our current development can be im- proved in future research. First, in our approach, reliable


TABLE 111-A NUMERIC RESULTS OF COMPUTER SIMULATIONS A-BASED ON f,' AND NO RECOGNITION KNOWLEDGE

Sensor 1 Sensor 2 Entre, Proc. Reloc. Candidates

Asgn. Location Asgn. Location Time Cost Left Iter.

0 8.0, 2.0, 10.0 7.0 0.0, 4.0 1 d i 5.5, 0.0, 3.5 4.0 0.75 2.37 1, 2, 3, 5 , 6, 7, 8 2 (1 I 5.5, 0.0, 3.0 3.5 1.0 0.75 1 , 2 , 3 , 5 , 7 , 8 3 d3 3.5, 0.0, 3.0 3.0 0.75 3.0 1, 3, 8 4 '1.1 7.5, 6.0, 10.0 1.5 1.5 4.84 1, 3 5 d9 3.5, 6.0, 10.0 1 .0 1.0 4.8 1

TABLE 111-B NUMERIC RESULTS OF COMPUTER SIMULATIONS B-BASED ON f,' AND Ii,j

Sensor 1 Sensor 2 Proc. Reloc. Candidates

Asgn. Location Asgn. Location Entre. Time Cost Left Iter.

3.0, 0.0, 9.0 0 6.0, 5.0, 10.0

1 d:$. d.$. d=, 6.0, 6.0, 10.0 6.2 3.5 1.2 1, 3 2 dc3 3.5, 6.0, 10.0 1.0 1.0 3.0 1

TABLE IV COMPARISON OF THE PERFORMANCE OBTAINED BASED ON f,' A N D ff

Ave. number of Ave. number of Ave. number of iter. discr. discr./iter. Ave. proc. time Ave. reloc. cost

f,' 3.62 3.88 f' 2.38 4.38

1.12 2.04

3.94 10.56 5.25 7.58

TABLE V COMPARISON OF THE PERFORMANCE OBTAINED WITH AND WITHOUT I i A

~

Ave. number of Ave. numvber of Ave. number of iter. discr. discr./iter. Ave. proc. time Ave. reloc. cost

without I i q 2.5 4.5 with I<? 2.0 3.5

eliminations of invalid candidates from a CADIG substantially relies on an appropriate measure of D-S beliefs in noisy sensory examinations of discriminators. The derivation of this measure was not covered in this paper and only a simple measure was used in computer simulations. Second, the computation of entropies for identifying critical information enlightens a feasible and promising approach to knowledge- based recognition of objects. This requires further exploration of the employed belief-based mechanism in future research. Third, the formulation of coordination of sensors for recognition as a CSP does not consider the fusion of sensory observations. This simplification, to some extent, constrains the advantages that would be obtained from using multiple sensors for recognition. Lastly, a laboratory experiment is desired to test the real-time performance and further analyze and improve the computational complexity of the proposed strategic recognition of objects.

1.83 5.50 7.05 1.83 4.25 7.05

APPENDIX

Algorithm ODP determines a completeness-preserving subset of discriminators, D*, from a CADIG with the minimum total sensing cost. Let Q = (C.A,@,4d) be a CADIG and 4c : 0 + R+ be the cost function that assigns each discriminator d E (3 a sensing cost 4c (d ) . Initially, a CDT, containing only the root node that is attributed with C , is given as the global database at the starting node no of the search graph. Any node encountered on the search graph is a goal node if its corresponding global database (CDT) is complete. And any two nodes on the search graph are treated as identical if the sets of candidates on the leaf nodes of their corresponding CDT are equivalent set by set.

Algorithm ODP: (Optimal Discriminator Pruning). S1: [Skeletonization of 9.1 Reduce the set of discriminators

associated with each arc in Q , if not empty, to contain

TANG AND LEE: OPTIMAL STRATEGIC RECOGNITION OF OBJECTS 66 1

only one discriminator in the set that is observable to 171 R. C. Luo and M. G. Kay, “Multisensor integration and fusion in . _ at least one sensor with the minimum sensing cost. intelligent systems,” I E E i Trans. Syst., Man, kybern., vol. SMC 6,

pp. 257-266, May 1984. 181 W. E. L. Crimson, “The combinatorics of local constraints in model- s2: [Initialization.] Create a set OPEN containing . .

only one node no. Also create a set called CLOSED based recognition and localization from sparse data,” J . Assoc. Comput. Machinery, vol. 33, no. 4, pp. 658-686, 1986.

191 W. E. L. Grimson, “Sensing strategies for disambiguating among mul- t ide obiects in known Doses.” IEEE J . Robotics Automat.. vol. 2.

that is initially empty. S3: [Select the best node in OPEN for expansion.] Select

the node in OPEN with the minimum f ( 7 1 ) . If several nodes qualify, choose a goal node if one exists; otherwise choose it arbitrarily. Call this node 71,. Remove

p i . 1961213, 1986. [ 10) S. A. Hutchinson, R. L. Cromwell, and A. C. Kak, “Planning sensing

strategies in a robot work cell with multi-sensor capabilities,” in Proc. 1988 IEEE Int. Conf Robotics Automat., Philadelphia, PA, Apr. 24-29,

node T L from OPEN and add it to CLOSED. S4: [Check goal node.] If n is the goal node, exit with

success, and D* is given by the set of discriminators encountered while tracing the pointers from n to n o ; otherwise continue.

S5: [Expand node n.] The node n can be expanded by any discriminator d E D,, where D,, is the set of discriminators in 0, which can further expand the CDT at n. Expand node n by each d E D,, to a successor node n,. Do steps S6 to S8 for each successor 7 1 , .

S6: [Successor ns not in OPEN or CLOSED.] When 71, # OPEN and n, CLOSED, mark a pointer from n, to n, compute f (n , ) according to (3) and (4), and add n, to OPEN.

S7: [Successor 7 ~ , in OPEN.] When ‘n,s E O P E N , if g ( n ) + & ( d ) < g ( n , ) , redirect the pointer from T L , ~

to n and compute f (n , ) according to (3) and (4). S8: [Successor n,s in CLOSED.] When ~ 1 , ~ E CLOSED, if

g ( n ) + 4 c ( d ) < g(n,s), redirect the pointer from n, to U, compute f (n ,s) according to (3) and (4), and check each descendant node of n, for redirection of the pointer if necessary.

S9: [Loop.] Go to step S3. END ODP.

REFERENCES

[l] A. P. Reeves and R. W. Quong, “Identification of three-dimensional objects using range information,” /€E€ Trans. Pattern Anal. Machine Intell., vol. 11, pp. 403-410, 1989.

[2] M.J . Magee, B.A. Boyter, C.H. Chien, and J.K. Agganval, “Experiment in intensity guided range sensing recognition of three- dimensional objects,” /€€€ Trans. Pattern Anal. Machine Intell., vol. 7, pp. 629-637, 1985.

[3] R. A. Browse, “Feature-based tactile object recognition,” I€€€ Trans. Pattern Anal. Machine Intell., vol. 9, pp. 779-786, 1987.

[4] P. C. Gaston and T. Lozano-Perez, “Tactile recognition and localization using object models: The case of polyhedra on a plane,” /E€€ Trans. Pattern Anal. Machine Intell., vol. PAMI-6, pp, 257-266, May 1984.

[5] W. E. L. Crimson and T. Lozano-Perez, “Model-based recognition and localization from sparse range or tactile data,” Int. J . Robotics Research, vol. 3, no. 3, pp. 3-35, Fall 1984.

[6] D. Nitzan, “Three-dimensional vision structure for robot applications,” /€E€ Trans. Pattern Anal. Machine Intell., vol. PAMI-10, no. 3, pp. 291-309, May 1988.

1988, pp. 1068-1075. [ 111 G. K. Cowan and P. D. Kovesi, “Automatic sensor placement from

vision task requirements,” IEEE Trans. Pattern Anal. Machine Intell., vol. 10, pp. 407-416, 1988.

[ 121 G. Beni, S. Hackwood, L. A. Hornak, and J. L. Jackel, “Dynamic sensing for robots: an analysis and implementation,” Int. J . Robotics Res., vol. 2, no. 2, pp. 51, 1983.

[ 131 H. F. Durrant-Whyte, Integration, Coordination and Control of Multi- Sensor Robot System.

1141 Y.C. Tang and C.S. G. Lee, “A geometric feature relation graph formulation for consistent sensor fusion,” I€EE Trans. Syst., Man, Cybern., vol. SMC-22, pp. 115-129, Jan./Feb. 1992.

[ 151 S. Umeyama, “An eigen-decomposition approach to weighted graph matching problems,” IEEE Trans. Pattern Anal. Machine Intell., vol. 10, pp. 695-703, 1988.

[16] L. G. Shapiro, J. D. Moriarty, and R. M. Haralick, “Matching three- dimensional objects using a relational paradigm,” Patfern Recog., vol. 17, no. 4, pp. 385-405, 1984.

[ 171 D. W. Thompson and J. L. Mundy, “Three-dimensional model matching from an unconstrained viewpoint,” in Proc. 1987 I€€€ In/. Conf Robotics Automat., 1987, pp. 208-233.

I 181 J . Pearl, Probabilistic Reusoning in Intelligent Systems: Networks of Plausible Inference.

[ 191 G. Shafter, A Mathemutical Theory of Evidence. Princeton, NJ: Prince- ton Univ. Press, 1976.

[X)] A. P. Dempster, “A generalization of Bayesian inference,” J . Royal Statist. Soc., ser. B 30. pp. 205-247, 1968.

1211 N. J. Nilsson, Artificial Intell.. 122) J. Pearl, Heuristics, Addison-Wesley, 1984. vol. 8, pp. 99-1 18, 1977. I231 T. Henderson, “Logical sensors,” in Proc. 1988 IEEE Int. Conf Robotics

Automat., Philadelphia, PA, Apr. 24-29, 1988, pp. 1246- 1253. [24] Y. C. Tang, “Integrated sensors in robotic assembly tasks,” Ph.D.

dissertation, School Elec. Eng., Purdue Univ., West Lafayette, IN, Dec. 1990.

[25] A. K. Mackworth, “Constraint Satisfaction,” in Encyclopedia ofArtiJicial Intelligence.

1261 A. Mackworth, “Consistency in networks of relations,” Artificial Intell.,

1271 R. M. Haralick and L. G. Shapiro, “The consistent labeling problem: Part I,” IEEE Trans. Pattern Aiial. Machine Intell., vol. PAMI-1, pp. 173-184. Apr. 1979.

Norwell, MA: Kluwer Academic, 1988.

San Mateo, CA: Morgan Kaufmann, 1988.

San Mateo, CA: Tioga, 1980.

New York: Wiley, 1987, pp. 205-211.

vol. 8, pp. 98-118, 1977.

Y.C. Tang, for a photograph and biography, please see page 129 of the JanuaryiFebruary 1992 issue of this TRAN5ACTIONS.

C.S. George Lee (S’71-S’78-M’78-SM’86), for a photograph and biography, please see page 129 of the JanuaryiFebruary 1992 issue of this TRANSACTIONS.

Documents

Optimal strategic recognition of objects based on candidate discriminating graph with coordinated sensors