An intersection property for starshaped sets in the plane

MARILYN BREEN*

AN I N T E R S E C T I O N P R O P E R T Y F O R S T A R S H A P E D SETS

IN T H E P L A N E

ABSTRACT. Let ~- be a family of compact starshaped sets in the plane. If every three and every two members of ~ have a union which is connected and simply connected, then N {F: F in ~ } is simply connected and nonempty. Of course, if every three and every two members of ~- have a starshaped union, the same result holds.

1. INTRODUCTION

We begin with some familiar definitions. Let S be a subset of R d. For points x and y in S, we say x sees y via S if and only if the associated segment [x, 3:] lies in S. Set S is called starshaped if and only if there is some point p in S such that p sees via S each point of S, and the collection of all such points p is the (convex) kernel of S.

Various types of combinatorial results have been established for star- shaped sets. There is a well-known characterization theorem by Krasnoserskii I-4] which states that for S a nonempty compact set in R d, S is starshaped if and only if every d + 1 points of S see via S a common point. Concerning starshaped unions of sets, Ko~odziejczyk [3] has proved that for ~,~ a finite family of closed sets in R a, if every d + 1 members of ~ have a starshaped union, then U{F: F in ~ } must be starshaped as well. The relationship between starshaped unions and nonempty intersections of convex sets, investigated in [1], provides the following Helly-type result: For f9 a nonempty family of compact convex sets in R d, every subfamily of consisting ofd + 1 or fewer sets has a starshaped union if and only if N{G: G in f#} # ~ . Here the family f# will be replaced by a family ~ of compact starshaped sets in the plane. A theorem of Moln~tr [6], [7] will be used to show that if every three and every two members of ~- have a sharshaped union, then N{F: F in ~ } is simply connected and nonempty.

Throughout the paper, conv S, cl S, int S, bdry S, ker S, and --~ S will denote the convex hull, closure, interior, boundary, kernel, and complement, re- spectively, for set S. For distinct points x and y, (x, y) will denote the open segment joining x and y, L(x, y) will be the line they determine, while R(x, y) will be the ray emanating from x through y. Finally, dist will represent the

* Supported in part by NSF grants DMS-8705336, DMS-8908717 and by a Senior Faculty Summer Research Fellowship, Research Council, University of Oklahoma.

Geometriae Dedicata 37: 317-326, 1991. © 1991 Kluwer Academic Publishers. Printed in the Netherlands.

318 M A R I L Y N B R E E N

Euclidean metric. The reader is referred to Valentine [8] and to Lay [51 for a

discussion of related concepts and to Danzer, Gr i inbaum and Klee [2] for a

survey of Helly-type theorems.

2. THE RESULTS

The following theorem by Moln~r [61 [7] will be needed.

MOLN~tR's T H E O R E M . A family ~ of at least three simply connected compact sets in R 2 has nonempty, simply connected intersection provided each two have connected intersection and each three have nonempty intersection.

T H E O R E M 1. Let S be a compact starshaped set in the plane, with x a point in kerS. Let R 1, R 2 be distinct rays from x, and let U be one of the two closed subsets of the plane whose boundary is R l u R 2. Then T - cl[(bdry S) n U ~ (xi] is connected.

Proof. Assume that T is nonempty, for otherwise there is nothing to show.

Notice that T is either (bdry S) n U or (bdry S) n U ~ (x}. To accomplish the

proof, we assume that set T is disconnected, to reach a contradiction. Then T may be expressed as a union of separated sets A and B. Clearly A and B are

closed. We consider two cases.

CASE 1. Suppose that x 6 T. Then T = (bdry S) n U ,~ {x}. For any ray R

from x such that R _c U, we assert that either R n A = ~f or R n B = ~ . If

a e R n A and b ~ R n B, then, without loss of generality, assume the order on

R is b < a < x. This implies that (a, b) n i n t S = ~ , for otherwise a could not

be a boundary point of S. Since S is starshaped at x, ( a , b ) _ S, so

(a, b ) _ bdry S. It follows that [a, b] is a connected subset of T. However,

I-a, b] is a union of the separated sets [a, b] n A # O and l-a, b ] n B 4: ~ .

This is impossible, and our assertion is established. Next we show that for every ray R from x with R _~ U, R ~ {x} meets

bdry S and hence meets A or B: Since x $ T, there is a convex neighborhood N

of x with N n (bdry S ) n U ~ {x} = ~ . Choose y ~ ( b d r y S ) n U ,~ {x} # J~.

Then

[ x , y ] ~ _ S and (x,y) n N ~ N n U ~ { x } ~ i n t S .

Thus (N n U ~ {x}) n (int S) # ~ . Since N n U ~ {x} is polygonally connected and disjoint from bdryS,

standard arguments show that N n U ~ {x} _ int S. Therefore, every ray R

described above does indeed meet A or B. For convenience, suppose R 1 ~ {x} meets A (and therefore not B). Then

since A and B are closed, there must be a ray Ra from x, R s _ U, R 3 # Rt,

I N T E R S E C T I O N P R O P E R T Y FOR STARSHAPED SETS 319

such that Ra ~ {x) meets B and such that the angle determined by RI and R3 has minimal measure. If V is the subset of U whose boundary is R~ u R3, then (V ~ R3)n B = ~ , and every ray from x in V ~ R3 necessarily meets A. However, this forces R3 to meet cl A = A, contradicting the fact that no such ray meets both A and B. Case 1 cannot occur.

CASE 2. Suppose that x E T, and for convenience let x E A. Then x is not in the closed set B, so dist(x, B ) - ~ > 0. We assert that not every ray from x in U meets B: Otherwise, for M the t$-neighborhood of x, M n U ~ S, and M n i n t U ___ intS. Since xeT , for at least one of R1 or R2, say R~, there would be a sequence in (R1 ~ ( x ) ) n A n M converging to x, so Rx n S n A ~ ~ . Since x e k e r S , it is not hard to see that RI n S ___ bdryS and that Rt n S is connected. We are assuming that every ray from x in U meets B. This forces the connected set R1 n S to be the union of the separated sets R t n A and R1 n B, which is impossible. Our assertion is established.

Let Ro be some ray from x in U which fails to meet B. As in Case 1, select a ray R3 from x in U such that R3 meets B and such that the angle determined

by Ro and R3 has minimal measure. Let V be the subset of U whose boundary is Ro u R 3. Select point b EB n R3 with dist(b,x) minimal, and choose a convex neighborhood N of b disjoint from A. Then both set N n V ~ R3 and set (x, b )n N are disjoint from A u B. Their union is connected and (by standard arguments) must lie in intS. Moreover, for each point z e N n V ~ R3 _cint S, there is some boundary point z' of S on R(x, z), so z 'e R(x, z) ~ [x, z]. Certainly z ' e A. Hence there is a sequence of points in A converging to a point a of (R3 ~ Ix, b]) n A. By an earlier argument from Case 1, [a, b] ~ bdry S. Thus [a, b] is a connected subset of T. However, [a, b] is also the union of the separated sets [a, b] n A ~ O and [a, b] n B ~ ~ . Again we have a contradiction, and Case 2 cannot occur either.

Our original assumption must be false, and set T is connected, completing the proof.

COROLLARY 1. Let S, x, RI, R2, U be the sets in Theorem 1. Then cl[bdry S n i n t U] is connected.

Proof. Assume int U ¢ ~ , for otherwise there is nothing to prove. Let

{R1, } and {R2n } be sequences of rays at x in (int U) u {x} converging to g 1 and R2, respectively. Let Un denote the closed subset of U whose boundary is RI~ u R2n, n/> 1. By Theorem 1, Tn -= c l [ (bdryS)n Un ~ {x}] is connected for each n. Certainly no two T~ sets can be separated, so by a familiar result concerning connected sets, U{T~:n>~ 1} is again connected. However, U{T~: n ~> l} is exactly cl[bdry S n i n t U].

320 M A R I L Y N B R E E N

COROLLARY 2. I f S is any compact starshaped set in the plane, then bdry S is connected.

Proof If S is a single point, the result is trivial. Otherwise, S has at least two (in fact, infinitely many) boundary points. Let x ~ ker S, y e S ~ {x}, and let R~ and R2 be distinct rays at x, with y~R~. Let U, V denote the closed subsets of the plane having boundary RI u R 2. Since the sets T, = cl[(bdry S) c~ U ,-, {x}] and T v = cl[(bdry S) c~ V ~ {x}] are connected and overlapping, T, u T~ = bdry S is connected, also.

THEOREM 2. Let FI and F a be compact starshaped sets in the plane whose union F I u F 2 is simply connected. Then F: n F 2 is connected.

Proof If F~ c~ F 2 is a singleton set, the result is immediate. Otherwise, let distinct points s and t belong to F I n F2. We must show that s and t belong to the same component of F~ c~ F 2. Select xi~ker F i, i = 1, 2. In case both x 1 and x2 are on L(s, t), then [s, t] __. F~ c~ F2, finishing the proof. Hence for the remainder of the proof, we may assume that x:q~L(s,t) and dist(x~, L(s, t)) >>. dist(x2, L(s, t)). Similarly the proof is trivial when xl = x2, so assume xl ~ x2.

There are various cases to consider, depending on the positions of x~ and

X 2 •

CASE 1. Assume that xl and x2 lie in the same closed halfplane determined by L(s, t) and that x2 is not in the convex subset U~ of the plane whose boundary is R(xl, s) u R(xl, t). Then (for an appropriate labeling of s and t) (x2,s) c~ (x~, t ) ~ ~ . Let t' be this point of intersection. Since F~ w F2 is simply connected, conv{x:, s, t'} w conv{x2, t', t} _ F1 w F2. (See Figure 1.)

We will show that s and t' are in the same component of F~ c~ F 2. Define S~ = F~ n conv{x~, s, t'}. Then S~ is a compact, starshaped subset of F1, and xl e kerS1. By Theorem 1, Corollary 1, T1 = cl(bdryS1 h i n t U~) is con-

x I

F ig . 1.

I N T E R S E C T I O N P R O P E R T Y FOR S T A R S H A P E D SETS 321

nected. Observe that T t __. bdry St _c Ft. We assert that Tt -~ F2, also: Let

webdryS tn in tU1. If we(s,t'), then certainly w e F 2. If w¢(s,t'), then w e bdry $1 c~ int cony{x1, s, t'} _ bdry Ft. Since int conv{xt, s, t'} ~ Ft u F2, there must be arbitrarily small neighborhoods of w in F t u F2. Because w¢intF1, every such neighborhood meets F2, so w e c l F 2 = F 2. Hence bdryS t n i n t U~ _~ F 2, and since F 2 is closed, T1 = cl(bdry S~ n i n t U1) _~ F 2 as well. The assertion is established. We have Tt ~ F1 n F2.

For the moment, assume Tt # ~ . We will show that Tt, s, and t' lie in the same component o f F 1 c~ F 2. This is obvious ifs and t belong to T t. I f s¢ T~, we will extend 7"1 to a connected subset of F~ c~ F2 which contains s.

First we show that I-s, x l ] n Tt # ~ : Otherwise, standard arguments yield a convex neighborhood N of l-s, xa] disjoint from T t and hence disjoint from bdryS~ c~int Ut. That is, N h i n t Ut is disjoint from bdrySt , so either

N c ~ i n t U t ~ S t or NnintUl___, -~St . If N c ~ i n t U t ~ S 1 , then N n (s, t') ~_ bdry S~ c~ int Ut ~ T1, impossible since N n Tt = O. Thus N n i n t U~ _ " S t c~ int U1 - ~ F t . Since xx e N n ker F1, this implies that F~ n i n t U~ = ~ , so S1 c~ int Ut = ~ and 7"1 = ~ , impossible since we are assuming that T~ ~ ~ . We have a contradiction and Is, x t ] n 7"1 ¢ ~ .

Let So be the point of Is, x l ] n 7"1 closest to s. Since Is, so) c~ Ta = ~ , for every s' e [s, so), [s, s'] n T1 = ~ . Moreover, for N' an appropriate neighbor-

hood of [s, s'],

N ' n i n t U _~ ~T~ n i n t U ~ ~F~.

Since int conv{xl, s, t'} ~ F l u F2,

this implies that N' c~ int conv{xl, s, t '} _~ F2, [s, s'] _ F2, and Is, So] - F2. Hence [s, So] ~ F1 c~ F2, and [s, So] u 7"1 is a connected subset of F1 n Fz which contains s.

In case t' ¢ T~, we apply a parallel argument to obtain point to closest to t'

in [t', Xl] c~ 7"1. Then 7"1 w [t', to] will be a connected subset of F1 n F2 containing t'. We conclude that s, 7"1, t' all lie in the same component of

F1 c~ F2, the desired result. If 7"1 = ~ , arguments like the ones above show that

Es, x~] u [x l, t'] ~ Fx c~ F2,

so s and t' again lie in the same component of F t n F2. Now if t = t' (that is, if x2 e L(s, t)), then the proof is completed for Case 1. If

t ~ t', then the argument may be repeated to show that points t' and t belong to the same component of F~ c~ F 2. Combining this with our result for s and t', we conclude that s and t are in the same component of F1 n F2, finishing

Case l.

322 M A R I L Y N B R E E N

x 1

Fig . 2.

CASE 2. Assume that Xl and x2 lie in opposite open halfplanes deter- mined by L(s, t) and that x2 is not in the convex subset Ux of the plane whose boundary is R(Xl, S)u R(Xl, t). For an appropriate labeling of s and t, R(xl, t ) ~ [Xl, t] meets (x2,s) at point t', and conv{xl, s, t} ___ conv{xl, s, t'} ~ F1 u F2. (See Figure 2.) The argument from Case 1 may be applied to $1 = FI c~ conv{x~, s, t'} to produce T~ = cl(bdry $1 c~ int U~), a connected subset ofF1 n F2. If Tx # ~ , then (if necessary) it may be extended to a connected subset of Fx n F2 which contains s and t. If T1 = ~ , then by an argument in Case 1, I-s, x~] u I-x1, t] ~ Fx n F2. Either way, s and t are in the same component of F 1 c~ F 2.

CAS~ 3. Assume that x2 is in the convex subset U1 of the plane whose boundary is R(xl, s) u R(xx, t). In case x2 ~ bdry UI, then a simplified version of the following argument finishes the proof, so we restrict attention to the case for Xl t i n t U1. (See Figure 3.)

Certainly the simply connected subset of the plane bounded by Is, xl] u Ix1, t] u [t, x2] u [x2, s] lies in F1 u F2 and contains Ix1, x2]. Moreover, standard arguments produce a point t 'e F~t~f 2 ~ l-X l , X 2 ] . The

x 1 x I

x 2

Fig . 3.

INTERSECTION PROPERTY FOR STARSHAPED SETS 323

argument from Case 1 may be applied to F1 n cony{x1, s, x2} to produce a connected subset of F1 n F2 containing s and t'. A parallel argument for F1 n conv{xDt, x2} yields a connected subset of F 1 n F2 containig t' and t. Again s and t are in the same component of FI n F2, finishing Case 3 and completing the proof of Theorem 2.

THEOREM 3. Let F~, F2, F a be compact starshaped sets in the plane. Assume that all three and every pair have a connected, simply connected union. Then

F1 n F 2 n F 3 # ~ . Proof. For 1 ~< i, ] ~< 3 and i # j , since F i w Fj is connected, certainly

F i n Fj # ~ . Select xi ~ ker Fi and choose a u ~ Fi c~ Fj. Then [xi, a u] -~ Fi, and the polygonal path

= IX 1, a12 ] u [a12, x2-] k.J Ix2, a23] u [a23, x3] t..., Ix3, a13 ] u [a13, Xl]

lies in F~ u F 2 u F 3. For convenience, define 22 = [al2, x2] w Ix2, a23] ~ F2 and 23 = [a23, x3] w [x3, a13] c F3.

Without loss of generality, assume that a23 is the only point in 42 n 23. (Otherwise, we may delete a connected subset of 22 u 23 on a closed curve and replace a23 with another point of 22 n 23 to achieve this property.) If a23 ~ FI, then a23 e F~ n F2 n F3, finishing the proof. If a23 ¢ FI, choose the point P12 of 22 n F~ closest to a23 along 22, and let 4[ be the subpath of 4 2 from a23 to P12- Likewise, choose the point Pl3 of 23 t~ F 1 closest to a23 along 23, and let 4~ be the subpath of 43 from a23 to Px3. Let 4~ = [P12,xx] u [xl,p13] ~- F1. (See Figure 4.)

Then 4' = 4[ w 4[ w 4~ is a closed curve contained in the simply connected set F~ u F 2 w Fa. Hence the region A bounded by 4' is also a subset of F 1 u F 2 u F 3. Observe that if the region is degenerate, then 2~, 4[, 4~ determine compact, collinear segments. Since every two of these segments

P12 ~ . l ~ /~'23 --'13

x 2 Fig. 4.

324 M A R I L Y N BREEN

intersect, they all intersect, and F 1 c~ F 2 n F3 ~ ~ . Therefore, we may assume that the region A is not degenerate. Moreover, assume P12 ~ P13, for otherwise the proof is finished since p~ is in 2j ~ F t n Fj f o r j = 2, 3.

For now, assume PI2~(Pla, x 0 and Px3q~(Px2, Xx). Then there are two edges of 2' at x I which meet only at Xl. Let R12, R13 denote the rays from xl

determined by these edges, with p~2eR~2 and p13eRx3. Let U denote the closed subset of the plane whose boundary is R12 u R~3 and for which every open ray from xt in U meets 2~ u 2~. Let A = cl(bdry F~ c~ int U) c_ F~. By Theorem 1, Corollary 1, A is connected.

For the moment, assume A ~ ~ . Observe that for each Co e

bdry F t n i n t U, Ix1, Co] ~ F1. Furthermore, R(xl, Co) meets 2~ u 2~, say at point Po, where Po is chosen as close as possible to c o. By our choice ofp~ 2 and P13, certainly Po ~ F1. Hence the points of R(xx, Co) may be ordered so that

Po < Co < x~. Moreover, since co ebdry FI, [Po, Co] c~ intFx = ~ . By our choice of Po, (po, xl) lies interior to the region A, and therefore

(Po, xl) - int(Fl u F 2 u F3). We have (Po, Co) - int(F1 u F 2 L) F3) ~ (int F1), SO (PO, Co) c F 2 U F 3. Thus Co ~ F2 L) F 3 and A _ F 2 k3 F 3 .

We will extend set A to a connected set A' _ F 1 c~ A n ( f 2 U F3) such that P~2, Pl3 cA'. There exists a sequence of points {p.} in (2~wA~)c~intU converging to a point p in R12. Moreover, {p.} may be chosen so that p is as close as possilbe to Px2 o n R12. For each p., there is an associated point c, ~ l-x1, p j n bdry F1, and some subsequence of {c.} converges, say to point c. Clearly ceAc~F~ and p ~< c ~< x~. We assert that [p~2,c] c_ A: By our choice of P~2 and P~3, it is not hard to see that we cannot have P12

< p < x~. I fp ~< PI2 ~< c ~< .x 1, then certainly [p12 , C'I c_. [-P12, X1 ] ~ /~Pl, and [p~,c] ~ cl intA. Otherwise, p ~< c <~P~2 ~< Xx. By our choice of {p,}, if P ~ P~2, then no point of(p, Pt2] is the limit of a sequence in (;t~ ~ 2~) c~ int U.

Hence [c, P12] - [P,P~2] - clint A. By a parallel argument, we obtain a point c ' eR lac~A with

[p~3, c'] _ clint A. Since A is connected and c, c 'eA, the set A ' = - [pxz, c ]~A~[c ' , p~3 l is connected as well. Moreover, it is easy to see that A' ~ F~ c~ A.

We will show that A' ~ F2 w F3: I fp ~< c < P~2 ~< x~, then by our choice of ply, points of ).~ sufficiently near P~2 must not be in int U. Similarly, since (c, P~2) - F~, again by our choice of P~2, points of 2'~ near p~z cannot be on R12 either. Thus x~ ~ P12, and points of 2~ near P12 are on the opposite side of R12 from int U, call it R~-2. For convenience, let s represent the correspond-

ing segment of 2~, with P12 es _ R~-2 ~ {P12}. (See Figure 5.) For each w ~ (c, Px 2), there is a sufficiently small neighborhood N of w such

that N ~ R~" 2 ~ A c_ F~ w F 2 U F 3 and such that for x in N c~ R~-2,

INTERSECTION PROPERTY FOR STARSHAPED SETS 325

Fig. 5.

[xi,x] n s # 0. Since s n F, = 0, it follows that x4 F,. Hence N n R12 n F, = Qj and N n R12 E F2 v F,. Therefore, (c,p,,) c F, v Fs,

and Cc, plzl E F, C-J FJ, the desired result. In case p d p12 < c < x1, then since (c,, p,) E int A E int(F, u F, u F3) and (c,, pn) n int F, = 0, (c,, p,) c_ F2 u I;,. Thus [c, p12] E [c, p] c F, u F,, again the desired result. Likewise, [c’,plJ c F, u F,, so A’ -C P2 u F,.

We have A’ a connected subset of F, n (F2 u F3). Clearly p12EA’nF2#0andp,,EA’nF,#0.HenceA’nF2nFs#@bypro- perties of connected sets. We conclude that

A’nF,nF,GF,nF,nF,#@,

finishing the proof in this case. If A = 0, then [pt2, x1] u [x,, plJ] is a connected subset of F, u F,

meeting both F2 and F,. Sin= Cp12, x,1 u Cx,, PA E F,, again F, n F2 n F3 # 0.

Finally, if p12 E(P~~, xi) or if p13~(p12, xi), then [p12, p13] is a connected subset of (F2 u F3) n F,, and once more F, n F, n F, # 0. This completes the proof of the theorem.

THEOREM 4. Let 9 be a family of compact starshaped sets in the plane. If every three and every two members of 9 have a connected, simply connected union, then n(F: F in 9} is simply connected and nonempty.

Proof. Every two members of 9 have a connected intersection by Theorem 2. Moreover, by Theorem 3, every three members of 9 have a nonempty intersection. Hence the result follows immediately from Molnar’s theorem.

THEOREM 5. Let 9 be a family of compact starshaped sets in the plane, and assume that every three and every two members of F have a starshaped union. Then every$nite subfamily of 9 has a starshaped union. Furthermore, n(F: F in S-) # 0.

326 'MARILYN BREEN

Proof. The first statement follows by applying Krasnosel'skii 's theorem to

U{Fi: Fi in ~-, 1 ~< i ~< n}. (See Kolodziejczyk [3, Th. 2, Cor.].) To see that N{F: F in ~ } ~ ~ , observe that ~- satisfies the hypothesis of Theorem 4.

In conclusion, [3, Example 3] may be modified to show that the restriction to finite subfamilies is needed in Theorem 5. Moreover, it is easy to find

examples which demonstrate that the simple connectedness and connected-

ness conditions in Theorems 2, 3 and 4 are required.

EXAMPLES. Let A, B, C denote distinct edges of a triangle in the plane. Sets

A u B and C show that simple connectedness is needed in Theorem 2, while sets A, B, C show that simple connectedness is needed in Theorems 3 and 4.

Letting C, D, E denote collinear line segments with C n D v~ ~ ,

D n E ~ ~ , and C n E = ~ , sets C, D, E demonstrate that connectedness is required for Theorems 3 and 4.

REFERENCES

1. Breen, Marilyn, 'Starshaped unions and nonempty intersections of convex sets in R~, ' Proc. Amer. Math. Soc. 108 (1990), 817-820.

2. Danzer, Ludwig, Griinbaum, Branko, and Klee, Victor, 'Helly's theorem and its relatives,' Convexity, Proc. Syrup. Pure Math., Vol. 7, Amer. Math. Sot., Providence, RI, 1962, pp. 101- 180.

3. Ko~odziejczyk, Krzysztof, 'On starshapedness of the union of closed sets in R"," Colloq. Math. 53 (1987), 193-197.

4. Krasnosel'skii, M. A,, 'Sur un crit6re pour qu'un domaine soit 6toi16,' Mat. Sb. (61) 19 (1946), 309-310.

5. Lay, Steven R., Convex Sets and Their Applications, Wiley, New York, 1982. 6. Molnfir, J., 'Ober den zweidimensionalen topologischen Satz yon Helly,' Mat. Lapok 8 (1957),

108-114. 7. Molnfir, J., 'Ober eine Verallgemeinerung anf die Kugelti/iche eines topoiogischen Satzes yon

Helly,' Acta Math. Acad. Sci. Hunoar. 7 (1956), 107-108. 8. Valentine, F. A., Convex Sets, McGraw-Hill, New York, 1964.

Author's address:

Marilyn Breen,

Dept. of Mathematics, University of Oklahoma, Norman, Oklahoma 73019,

U.S.A.

(Received. December 6, 1989)