Geometriae Dedicata 60: 283-288, 1996. 283 © 1996 KluwerAcademic Publishers, Printed in the Netherlands.

A Helly-type Theorem for Simple Polygons

MARILYN BREEN* University of Oklahoma, Norman, Oklahoma 730]9, U.S.A. e-mail: mbreen@uoknor.edu

(Received: 27 September 1994; revised version: 27 July 1995)

Abstract. Let P be a family of simple polygons in the plane. If every three (not necessarily dis- tinct) members of P have a simply connected union and every two members of P have a nonempty intersection, then N{P:P in P) ¢ ¢. Applying the result to a finite family C of orthogonally con- vex polygons, the set fq{C:C in d) will be another orthogonally convex polygon, and, in certain circumstances, the dimension of this intersection can be determined.

Mathematics Subject Classifications (1991): Primary 52A35; secondary 52A10.

Key words: Helly theorems, simple polygons, orthogonally convex polygons.

1. Introduction

Let A be a nonempty set in the plane. Set A is a simple polygon if and only if A is a connected, simply connected union of finitely many convex polygons (possible degenerate). Set A is an orthogonalpolygon (rectilinear polygon) if and only if A is a connected union of finitely many convex polygons whose edges are parallel to the coordinate axes. Set A is said to be horizontally convex if and only if for each x, y in A with [x, y] horizontal, it follows that [x, y] C_ A. Vertically convex is defined analogously. Set A is an orthogonally convex polygon if and only if A is an orthogonal polygon which is both horizontally convex and vertically convex.

Although various Krasnosel'skii-type theorems have been established for orthog. onal polygons which are starshaped via staircase paths ([1],[2],[3],[4]), no associ- ated Helly-type theorems have been obtained for a family C of orthogonally convex polygons. Indeed, [1, Example 4] shows that no finite Helly number exists for such families. However, with an additional requirement on certain unions of these sets, it turns out that such a theorem holds, not only for a family C of orthogonally convex polygons but, more generally, for a family 79 of simple polygons. Furthermore, with appropriate hypothesis, the dimension of the intersection [']{C:C in C } can be found as well. Various examples show that the results are best possible and that the hypothesis is required.

Throughout the paper, cl C, int C, and bdry C will denote the closure, interior, and boundary of set C, respectively. The reader is referred to Valentine [8], to Lay [7], and to Danzer et al. [5] for a discussion of Helly's theorem and related results.

* Supported in part by NSF grant DMS-9207019.

2 8 4 MARILYN BREEN

Figure 1.

X

6

Y

k2

2. An Intersection Theorem

We begin with a Helly-type theorem for families of simple polygons.

THEOREM 1. Let 7 9 be a family of simple polygons in the plane. I f every three (not necessarily distinct) members of 7 ) have a simply connected union and every two members of 79 have a nonempty intersection, then N { P : P in79} ~ 0.

Proof To begin, we show by induction on n that if the polygons P1 , . . . , P~ in 7 9 have a nonempty intersection P = P1 n . . . N P~, then P is polygonally connected. If n --- 1, the result is immediate. Assume that the result is true for k-member subfamilies of 79, k >_ 1, to prove for the (k + 1)-member subfamily {P1,. . . ,Pk+l}. Choose arbitrary points z , y in P = P1 n . . . n Pk+l to show that P contains a polygonal path from z to y. By our induction hypothesis there exist polygonal paths A1 and A2 in P1 N P2 N • .. N Pk and P2 n .- . N Pk n P~+I, respectively, joining z to y. Without loss of generality, suppose that A1 and Aa have been selected so that their intersection contains a minimal number of components. We may assume that A1 N A2 = {z, y}. Let D denote the simple polygon bounded by ),1U A2. Since each of the sets P2 , • • • , Pk, and P1U P k + l is simply connected, we conclude that D C (P2 N P3 N . . . n Pk) N (P1 U Pk+l). Let Q be the component of D N/91 containing path A1. Then Q is necessarily a simple polygon and is bounded by the union of A1 and a polygonal path (5 from z to y, where (5 and A1 ~ {x, y} are not necessarily disjoint or even distinct. (See Figure 1.)

Clearly (5 C P1 N . . - N Pk. We assert that (5 C Pk+l as well and hence (5 C P. Let z be an arbitrary point of (5. If z C A2, of course z C Pk+l. If z ~ ),2, then every neighbourhood of z intersects the closed set Pk+l and again z C Pk+l. Hence (5 is a polygonal path in P from z to y, and P is polygonally connected. This finishes the induction and establishes the preliminary result.

To prove that N { P : P in 7 9} # 0, it suffices to show that every finite subfamily of 79 has a nonempty intersection. Again we proceed by induction. By hypothesis, the result is true for 2-member subfamilies of 79. Assume that the statement holds

A HELLY-TYPE THEOREM FOR SIMPLE POLYGONS 285

×

R X3

Y ./ Figure 2.

for k-member subfamilies, k > 2, to prove for {P1 , . . . , Pk+l}. By our induction hypothesis we may select

z E A { P i : l < i < k + l , i 7 6 3 }

y E (-]{P/:I < i < k + 1 , i ¢ 2}

z E ('}{Pi:I < i < k + 1 , i # 1}.

By our preliminary result, every nonempty, finite intersection of P sets is polygo- nally connected. Thus there exist polygonal paths

),1 in N{Pi:I < i < k + 1,i ¢ 2, 3} fromx toy,

),2in ~{Pi : I < i < k + 1,i ¢ 1,3} f romxtoz ,

),3in [-]{Pi:I < i < k + 1,i ¢ 1,2} f romytoz .

Again, suppose that the ),i paths have been selected so that {)`i r? )`j:l _< i < j _< 3} has a minimal number of components. Assume that the corresponding curve )`l U ),2 U ),3 is simple. Let R denote the region bounded by this curve. Since the union P1 U P2 U P3 is simply connected, as are P4,..., Pk+l (when k > 3), it follows that

-~ ~ (-P1 U -]9 2 U P3) n P4 n . . - n / gk+ l .

Let Q be the component of R R/91 which contains A1. As in the proof of the preliminary result, Q is a simple polygon whose boundary is AI U 6, where 6 is a polygonal path from x to y (see Figure 2).

We will show that 6 contains a point of N{Pi:I _< i _< k + 1}. Clearly 6 C N{P/:I < i < k + 1,i 76 2, 3}.Foreach point s in ~, either s E )`2 U.,)`3 C_ P2UP3 or every neighborhood of s intersects the closed set P2 U/:'3 and again s E P2 U P3. Thus 6 _C P2 U P3. Since z E 6 rh P2 and y E 6 f?/93 and the sets 6, P2, and P3 are connected, we conclude that there is some point q in 6 rq P2 rq P3. Thus

286 MARILYN BREEN

q E ~{P i : I < i < k + 1} andN{Pi : l < i < k + l } # 0, completing the induction and finishing the proof of Theorem 1.

Remark. Example 4 in [1] shows that some requirement (like the one above concerning unions of sets) is needed for a Helly-type theorem on orthogonally convex and hence on simple polygons. Furthermore, it is easy to find examples which show that each of the numbers three and two in Theorem 1 is best possible. Finally, it is interesting to observe that an analogue of Theorem 1 holds with 'simple polygons' replaced by 'compact convex sets'. This version, in turn, is reminiscent of a well-known theorem theorem by Klee [6].

3. Applications for Orthogonally Convex Polygons

Using Theorem 1, we are able to obtain the following result.

PROPOSITION 1. Let C be a family of orthogonally convex polygons in the plane. If every three (not necessarily distinct) members of C have a simply connected union and every two members of C have a nonempty intersection, then N{ A:A inC} ~ 0. Furthermore, when C is finite, this intersection is another orthogonally convex polygon.

Proof The first statement is an immediate consequence of Theorem 1. To verify the second statement, it suffices to show that N{A:A inC} is connected when C is finite. However, this follows from the preliminary result in the proof of Theorem 1.

In certain circumstances, we are able to obtain Helly-type theorems for the dimen- sion of an intersection of orthogonally convex polygons.

PROPOSITION 2. Let C be a finite family of orthogonalIy convex polygons in the plane. If every three (not necessarily distinct) members of C have a simply connected union and every four members of C meet in at least a segment, then N { A:A in C } contains a segment as well. The number four is best possible.

Proof. By Proposition 1, we know that N{A:AinC} is nonempty and is an orthogonally convex polygon. Select point p in this intersection. We assert that ~{A:A inC} contains a segment at p: Otherwise, for an appropriate choice of sets Ai~ 1 < i < 4, inC, A1 would contain no segment from p to the east, A2 no segment from p to the north, A3 no segment from p to the west, A4 no segment from p to the south. Then A1 N A2 N A3 N A4 would contain p as an isolated point, impossible since A1 A A2 N A3 n A4 is an orthogonal polygon (hence connected) and contains a segment (by hypothesis). We have a contradiction, and ~ { A:A in C } necessarily contains a segment at each of its points.

The following easy example demonstrates that the number four in Theorem 2 is best possible.

EXAMPLE 1. For 1 < i < 4, let si be the segment in the plane from the origin to the point (1,0), (0,1), (-1,0) , (0,-1), respectively, and define Ai = [,]{~j:j ~ i}.

A HELLY-TYPE THEOREM FOR SIMPLE POLYGONS

,l'l ..... l L~. " k /

/ /

/ /

,s /

/

I 1 . . . . • I

I

IG, Z., I

L,

Figure 3.

287

Each Ai is an orthogonal polygon, every three of the Ai sets contain a common segment, yet n{Ai : l < i < 4} contains only the origin.

Unfortunately, there is no direct analogue of Proposition 2 to guarantee that an inter- section of orthogonally convex polygons contain a two-dimensional neighborhood, as Example 2 reveals.

EXAMPLE 2. For m > 2, le t ) , = U{Ai:I _< i _< 2ra} be a staircase path in the plane, and let Ci be the rectangle determined by consecutive segments )~i and hi+z, i odd, 1 _< i < 2m. (See Figure 3.) For k odd, 1 _< k < 2m, define Ak = U{Ci:i 7 ~ k} U ~. Then each A set is an orthogonally convex polygon, and every m - 1 of the A sets contain a corresponding C' set, which is two-dimensional. However, n{Ak} = )~, which contains no two-dimensional neighborhood. Since ra may be chosen arbitrarily large, no Helly number exists to guarantee that such an intersection contain a two-dimensional neighborhood.

However, with additional requirements on pairwise intersections of the sets, we have the following Helly-type result. Note that the sets need not be orthogonally convex.

PROPOSITION 3. Let d be a finite family of orthogonal polygons in the plane such that I = n { A : A i n d } ¢ ~. Iffor every A, B in d, A N B = cl int(A N B), then [ = cl int I as well.

Proof It suffices to show that for every point p in I , I contains a nondegenerate rectangle at p. Let si, 1 _< i _< 4, denote four segments with endpoint p, each parallel to a coordinate axis, ordered in a clockwise direction (as in Example 1 above). By our hypothesis, A n B = cl int(A n B) for all A, B in C. Hence every pairwise intersection A N/3 contains a rectangle at p. Observe that such a rectangle necessarily includes segments at p contained in two consecutive si sets (relative

2 8 8 MARILYN BREEN

to our clockwise ordering). Furthermore, for either i = 1 or i = 3, each A set contains a segment at p along si: otherwise, for both i = 1 and i = 3, some set Ai would fail to contain such a segment along si, and consequently A1 N A3 could contain such segments along s2 and s4 only, violating our observation above.

Without loss of generality, assume that each A set contains a segment at p along Sl and hence a rectangle at p along sl. Let L denote the line determined by Sl, with Lt and L2 distinct open halfplanes corresponding to L. It is easy to see that for one of these halfplanes, say L1, each A set contains a rectangle at p along Sl and lying in cl Ll: Otherwise, for i = 1,2, some set Ai would contain no such rectangle in cl Li, and As N A2 would violate our hypothesis. Therefore, each A set contains an appropriate rectangle at p, and the intersection I must, also, finishing the argument.

COROLLARY. Let C be a finite family of orthogonally convex polygons in the plane. If every three (not necessarily distinct) members of C have a simply connected union and A N B = cl int (ANB) ~ O for every A, B in d, then I = N{A:A inC} 0 is an orthogonally convex polygon and I = cl int I as well.

Acknowledgement

The author would like to thank the referee for extending Theorem 1, originally proved for orthogonally convex polygons, to the much more general result for simple polygons which appears here.

References

1. Breen, Marilyn: Dimensions of staircase kernels in orthogonal polygons, Geom. Dedicata 49 (1994), 323-333.

2. Breen, Marilyn: An improved Krasnosel'skii theorem for orthogonal polygons which are star- shaped via staircase paths, J. Geometry 51 (1994), 31-35.

3. Breen, Marilyn: AKrasnosel'skiitheoremfor staircase paths in orthogonal polygons, J. Geometry 51 (1994), 22-30.

4. Breen, Marilyn: Staircase kernels in orthogonal polygons,Arch. Math. 59 (1992), 588-594. 5. Danzer, L., Grtinbaum, B. and Klee, V.: Helly's theorem and its relatives, Proc. Sympos. Pure

Math 7 (1962), Amer. Math. Soc., Providence, R. I., pp. 101-180. 6. Klee, V. L., Jr: On certain intersection properties of convex sets, Canad. J. Math. 3 (1951),

272-275. 7. Lay, Steven R.: Convex Sets and Their Applications, Wiley, New York, 1982. 8. Valentine, F. A.: Convex Sets, McGraw-Hill, New York, 1964.