HANDBOOK OF TOPOLOGICAL FIXED P OINT THEORY · 5. Periodic points andbraid theory T. Matsuoka 171 6. Fixed point theory of multivalued weighted maps J. Pejsachowicz, R. Skiba 217

HANDBOOK OF TOPOLOGICAL FIXED POINT THEORY

Handbook of Topological

Fixed Point Theory

Edited by

R. F. Brown

University of California,

Los Angeles, U.S.A.

M. Furi

Department of Applied Mathematics ‘G. Sansone’,

Florence, Italy

L. Górniewicz

Juliusz Schauder Center of the Nicolaus Copernicus University,

Poland

and

B. Jiang

Department of Mathematics,

Peking University, Beijing, China

A C.I.P. Catalogue record for this book is available from the Library of Congress.

Published by Springer,

P.O. Box 17, 3300 AA Dordrecht, The Netherlands.

Printed on acid-free paper

All Rights Reserved

© 2005 Springer

No part of this work may be reproduced, stored in a retrieval system, or transmitted

in any form or by any means, electronic, mechanical, photocopying, microfilming, recording

or otherwise, without written permission from the Publisher, with the exception

of any material supplied specifically for the purpose of being entered

and executed on a computer system, for exclusive use by the purchaser of the work.

Printed in the Netherlands.

ISBN-10 1-4020-3221-8 (HB)

ISBN-10 1-4020-3222-6 (e-book)

ISBN-13 978-1-4020-3221-9 (HB)

ISBN-13 978-1-4020-3222-6 (e-book)

www.springeronline.com

TABLE OF CONTENTS

Preface vii

Chapter I. HOMOLOGICAL METHODS IN FIXEDPOINT THEORY 1

1. Coincidence theory3

2. On the Lefschetz fixed point theorem´ 43

3. Linearizations for maps of nilmanifolds and solvmanifoldsE. C. Keppelmann 83

4. Homotopy minimal periodsW. Marzantowicz 129

5. Periodic points and braid theoryT. Matsuoka 171

6. Fixed point theory of multivalued weighted mapsJ. Pejsachowicz, R. Skiba 217

7. Fixed point theory for homogeneous spaces – a briefsurveyP. Wong 265

Chapter II. EQUIVARIANT FIXED POINT THEORY 285

8. A note on equivariant fixed point theory287

9. Equivariant degreeJ. Ize 301

problems with variational structureS. Rybicki 339

L. Gorniewicz´

D. L. Goncalvesc

D. L. Ferrario

10. Bifurcations of solutions of SO(2)-symmetric nonlinear

vi TABLE OF CONTENTS

Chapter III. NIELSEN THEORY 373

11. Nielsen root theoryR. Brooks 375

12. More about Nielsen theories and their applicationsR. F. Brown 433

13. Algebraic techniques for calculating the Nielsen numberon hyperbolic surfacesE. L. Hart 463

14. Fibre techniques in Nielsen theory calculationsPh. R. Heath 489

15. Wecken theorem for fixed and periodic pointsJ. Jezierski 555

16. A primer of Nielsen fixed point theoryB. Jiang 617

17. Nielsen fixed point theory on surfacesM. R. Kelly 647

18. Relative Nielsen theoryX. Zhao 659

Chapter IV. APPLICATIONS 685

19. Applicable fixed point principlesJ. Andres 687

20. The fixed point index of the Poincar translationoperator on differentiable manifolds

741

21. On the existence of equilibria and fixed points of maps un-der constraintsW. Kryszewski 783

22. Topological fixed point theory and nonlinear differentialequationsJ. Mawhin 867

23. Fixed point results based on the Ważewski methodR. Srzednicki, K. Wójcik, P. Zgliczyński 905

Authors 945

Index 949

M. Furi, M. P. Pera, M. Spadini

PREFACE

Fixed point theory concerns itself with a very simple, and basic, mathematicalsetting. For a function f that has a set X as both domain and range, a fixedpoint of f is a point x of X for which f(x) = x. Two fundamental theoremsconcerning fixed points are those of Banach and of Brouwer. In Banach’s theorem,X is a complete metric space with metric d and f :X → X is required to bea contraction, that is, there must exist L < 1 such that d(f(x), f(y)) ≤ Ld(x, y) forall x, y ∈ X. The conclusion is that f has a fixed point, in fact exactly one of them.Brouwer’s theorem requires X to be the closed unit ball in a Euclidean space andf :X → X to be a map, that is, a continuous function. Again we can conclude thatf has a fixed point. But in this case the set of fixed points need not be a singlepoint, in fact every closed nonempty subset of the unit ball is the fixed point set forsome map. The metric on X in Banach’s theorem is used in the crucial hypothesisabout the function, that it is a contraction. The unit ball in Euclidean space isalso metric, and the metric topology determines the continuity of the function, butthe focus of Brouwer’s theorem is on topological characteristics of the unit ball,in particular that it is a contractible finite polyhedron. The theorems of Banachand Brouwer illustrate the difference between the two principal branches of fixedpoint theory: metric fixed point theory and topological fixed point theory. TheHandbook of Metric Fixed Point Theory, edited by Art Kirk and Brailey Sims andpublished by Kluwer in 2001, presented that portion of the subject and, in thiscompanion volume, we take up the other part of the fixed point story.

The classification of mathematical content is seldom easy. For instance, the dis-tinction between the metric and topological fixed point theories is far from preciseand it can be difficult to determine to which a specific topic belongs. In the sameway, although fixed point theory is generally considered a branch of topology,the influence of nonlinear analysis, and the related subject of dynamics, is so pro-found that much of fixed point theory could just as well be considered a partof analysis. The papers in this Handbook reflect the varied, and not easily clas-sified, nature of the mathematics that makes up topological fixed point theory.To impose some structure on its contents, the papers have been divided into four

viii PREFACE

“chapters”, each of which consists of papers that have something important incommon with each other.

The title of Chapter I, homological methods in fixed point theory, points outthe importance of algebraic topology, specifically homology theory, as the sourceof many of the mathematical tools used in fixed point theory. This chapter alsoillustrates the fact that topological fixed point theory is not just about the equa-tion f(x) = x. For instance, if the function f is multi-valued, taking points of Xto subsets of the same space, a fixed point is a point such that x ∈ f(x). The pio-neering work of Lefschetz was in the context of coincidence theory. For two mapsf, g:X → Y between closed orientable manifolds of the same dimension, a nonzerovalue of the homotopy invariant that Lefschetz introduced implies the existenceof a coincidence, that is, a point x ∈ X such that f(x) = g(x). Another modifica-tion of the fixed point equation is fn(x) = x where fn denotes the n-times iterationof a map f :X → X. A point x such that fn(x) = x is called a periodic point.The iterates of f constitute a discrete dynamical system on X and the periodicpoints can furnish important dynamical information. The influence of dynamicscan also be observed in the fact that such homogenous spaces as nilmanifolds andsolvmanifolds are the setting for several of the papers in this chapter.

Since homogeneous spaces are spaces of cosets, algebra plays an important rolein the study of maps on such spaces. Another way in which algebra impacts fixedpoint theory is through the study of equivariant maps. If a space is acted onby a group, then an equivariant map is one that respects the action. Chapter IIis devoted to the fixed point theory of equivariant maps and its application toanalysis.

Topological fixed point theory is often referred to as “Nielsen theory”. This ter-minology reflects the importance of the concepts introduced by Jacob Nielsen thatfurnish a homotopy invariant lower bound for the number of solutions to an equa-tion. All the papers in Chapter III contain the name of Nielsen, or of Weckenwho expanded Nielsen’s ideas, in their title. Again the objects of study are notjust fixed points. Coincidences, periodic points and fixed points of multivaluedmaps all make their appearance in this chapter, as do roots, the solutions tothe equation f(x) = a where f :X → Y is a map and a ∈ Y . Nielsen theory isparticularly interesting, and challenging, when the spaces are compact surfaces, assome of the papers in Chapter III demonstrate.

The substantial size, and content, of Chapter IV indicates the importanceof the applications of topological fixed point theory to nonlinear analysis anddynamics. Problems are formulated in terms or fixed or periodic points, coinci-dences and roots. The tools of fixed point theory are those of the previous chapters:the Lefschetz number, fixed point index, Nielsen number and, for root problems,

PREFACE ix

the topological degree. Again the fuctions considered may be multivalued as wellas single valued. However, a notable difference between the papers in this chapterand many of those earlier in the Handbook is the extention of the tools to moregeneral settings than those of the purely topological investigations. These powerfultools are then employed to obtain results about periodic solutions and solutionssatisfying boundary conditions and other constraints, for differential equations anddifferential inclusions.

We have not attempted a definition of the “topological fixed point theory”that is the subject of this Handbook; neither will we try to define precisely whata “handbook” is. A handbook contains information that will furnish the math-ematician reader, whether an established researcher or a graduate student, witha foundation in its subject and a guide to further study. An up-to-date handbookalso gives its readers a sense of the present state of the art and, ideally, offerssome clues as to where the subject will go in the future. But a handbook is nota textbook in which the reader starts on the first page expecting to find a com-plete and detailed exposition following a clearly indicated line of development thatextends to the very last page. Instead, the reader of a handbook is invited to viewits table of contents as a buffet from which to taste some items while perhapsconsuming others that seem particularly attractive. Each of the 28 authors whocontributed to this handbook was asked to do so because the editors consider thatperson an expert on the topic that he or she was invited to write about. The styleof presentation and level of mathematical detail was determined by the authors,based on their own mathematical taste and their judgment of the best way topresent their specialty.

This handbook is the sum of the contributions of its authors. It exists becausethese busy people were willing to expend a considerable amount of time and effortand we are grateful to them for doing it. We very much appreciate the help of theJuliusz Schauder Center of the Nicolaus Copernicus University in Toruń. MariuszCzerniak managed the collection of the papers and Jolanta Szelatyńska convertedthem into a uniform style. We are grateful for the support of our editors at Kluwer:Liesbeth Mol who initiated the project and Lynn Brandon who saw if through tocompletion and Marlies Vlot who supervised the producion of the handbook.

The Editors

CHAPTER I

HOMOLOGICAL METHODS IN FIXED POINT THEORY

It is well known that homology theory plays a fundamental role in fixed pointtheory or, more precisely, in topological fixed point theory. With the applicationof homology theory one can obtain:

(a) global fixed point theorems,(b) local fixed point theorems.

In 1923 S. Lefschetz proved a global fixed point theorem, now called Lefschetzfixed point theorem. This theorem is still studied by a number of mathematiciansand it is an important part of topological fixed point theory. The second importantpart of topological fixed point theory is mainly connected with local fixed pointproblems and it is called fixed point index theory or, in particular, topologicaldegree theory. Fixed point index theory was introduced by H. Hopf in the late1920s for maps on finite polyhedra and essentially developed by B. O’Neill in 1953.Note that a modern definition of this notion was presented by A. Dold in 1965.

Topological degree theory was initited by E. L. Brouwer in 1912 in the finitedimensional case and extended by J. Leray and J. P. Schauder in 1934 to theinfinite dimensional case.

The purpose of this chapter is to present, in the most general form, both theglobal and local cases studied in topological fixed point theory.

1. COINCIDENCE THEORY

Daciberg L. Goncalvesc

1. Introduction

This survey article analyzes the problem of minimizing the number of coinci-dence points of two mappings by deforming them through homotopies. We presentthe main invariants which have been used in the study of this problem, trying toillustrate with examples which give a better understanding of how to develop thetheory. We will stress the particular features of coincidence theory and very sel-dom will we refer to fixed point and root theory, which are of course related. Thereader will find enough material about these two topics elsewhere, including inthis handbook. We also include a very brief exposition of some topics that havebeen treated only very recently.

The paper is divided into six sections. In Section 2, the main problems arestated, a few generalities are discussed, and several examples are presented toillustrate which kind of result one might expect. In Section 3, we present theLefschetz–Hopf trace formula. We consider this formula in several contexts such as,for maps between closed orientable manifolds, for maps between compact manifoldswith boundary and for maps between nonorientable manifolds. Section 4 is devotedto the Nielsen coincidence classes and the Reidemeister coincidence classes. Theycan be defined in a quite general context without any conceptual difficulty. InSection 5, we deal with the index of an isolated set of coincidence points. Twotypes of index are analyzed, in particular when this set is a Nielsen coincidenceclass. This can be done in certain categories. We will consider the case where thedomain is a complex of dimension n and the target is a manifold also of dimensionn. In Section 6, we look at the case where the two spaces involved are complexesof dimension ≤ 2, in particular, the case of surfaces. There is a close relationshipbetween the coincidence problem on surfaces and the existence of solutions ofcertain equations in the braid groups. We will discuss these points in detail. InSection 7 we describe the recent work which has been done for maps between twomanifolds of different dimension.

4 CHAPTER I. HOMOLOGICAL METHODS IN FIXED POINT THEORY

The author wish to express his sincere thanks to Prof. R. Brown for his carefulreading and his valuable suggestions to make the exposition much clearer. Also toJ. Guaschi for helping with the presentation.

2. The coincidence problem

The development of coincidence theory has been greatly influenced by fixedpoint theory, although not only by it. It is fair to start this work by pointing outthe great influence of Brouwer’s Fixed Theorem, dated 1905, on the developmentof both fixed point theory and coincidence theory. Whereas the main emphasisof Brouwer’s result was on the existence of a fixed point, a little after, in the1920’s, Nielsen in his studies of surfaces (see [Ni]) was interested in estimatingthe minimal number of fixed points among all maps in a given homotopy classof maps. For this purpose an invariant was defined which we call nowadays theNielsen number. From 1923 to 1927, in a series of papers [L1]–[L4], Lefschetzgreatly generalized Brouwer’s result for coincidence under the hypothesis that thespaces involved were compact orientable manifolds of the same dimension. Thisis perhaps the beginning of coincidence theory. In 1929, Hopf in [H1] extendedLefschetz’s result for fixed points where the spaces in question were no longermanifolds but finite complexes. This is the famous Lefschetz–Hopf Theorem. In1936 Reidemeister described (in [R]) in an algebraic way (in terms of traces) notonly the information about the Lefschetz–Hopf number but the Nielsen classesand their indices, at least for fixed points. Wecken (see [We]) in the early 40’sshowed that for self maps of a complex satisfying mild conditions, the map canalways be deformed to have exactly the Nielsen number of fixed points. Thenin 1955, Schirmer (see [Sc]), published her thesis which showed a Wecken typeresult for coincidences of maps between two closed orientable manifolds of thesame dimension greater or equal to three. This was the first advance of the theoryafter Lefschetz. For more details about the above and also the early history ofthe subject, see the excellent article [Bw2] by Brown. Also for related topics ofcoincidence theory not covered in this article see [BGZ1]. I hope that the factsdescribed above give us enough motivation and information to understand themore recent development of coincidence theory.

We will consider the study of coincidence theory basically in two contexts. Thefirst one is for a pair of continuous maps (f, g):X → Y , and the second is fora pair of maps in the category of pairs of spaces. In some cases we will have toassume certain conditions, dividing the study into several subcases.

For the first case, we will assume in general that X is at least a finite CW-complex, and Y a (not necessarily orientable) manifold (without boundary). Thisis not the most general situation that one can imagine, and perhaps not even the

1. COINCIDENCE THEORY 5

one which covers all important applications. Nevertheless these cases reflect thesetting where a relevant and substantial amount of work has been done up tonow using algebraic and geometric topology. For other cases see [BGZ1]. Severalsubcases are of interest and they have their own features. Namely:

(2.1.1) X, Y are closed manifolds (compact and without boundary) of the samedimension greater than or equal to three.

(2.1.2) X, Y are closed manifolds of the same dimension equal to two.(2.1.3) X is either a finite complex or a finite CW-complex, and Y is a closed

manifold with dimX = dimY ≥ 3.(2.1.4) X is either a finite complex or a finite CW-complex, and Y is a closed

manifold with dimX = dimY = 2.(2.1.5) X, Y are compact manifolds where dimX ≥ dimY .(2.1.6) X is either a finite complex or a finite CW-complex, and Y is a closed

manifold with dimX ≥ dimY .

For the second situation, we consider the category of pairs of spaces (M1MM ,M2MM )where M1MM is a manifold and M2MM is a submanifold. Consider a pair of mapsf, g:M1MM → N1NN where the manifolds M1MM , N1NN have the same dimension. Coinci-dence theory may be studied under several different hypotheses on the maps f, gand the kind of submanifold. Few cases have been considered in the literatureso far. The one we will be most concerned with is the following: consider closedmanifolds with nonempty boundary and a pair of maps f, g:M → N where M,Nare manifolds of the same dimension. Assume that one of the maps, the map gfor example, is a map of pairs namely g: (M, ∂M) → (N, ∂N). In this category wedefine a homotopy to be a pair of homotopies ftff , gt, where ftff is a homotopy of f ,and gt is a boundary preserving homotopy of g i.e. gt(∂(M)) ⊂ ∂N . Coincidencetheory in this category was studied in [BSc1], [BSc2].

We never consider the situation where dimX < dimY . The reason is becauseunder this hypothesis there is no coincidence theory, see Proposition 2.9.

Let X, Y be two topological spaces and f :X → Y a continuous map. For apair of maps f, g:X → Y , let Coin(f, g) = {x ∈ X : f(x) = g(x)}. This setCoin(f, g) changes when we replace f , g by maps f ′, g′, respectively, where f ′, g′

are homotopic to f , g, respectively. Coincidence theory, generally speaking, studiesproperties relative to this family of sets. Here we try to present how algebraic andgeometric topology has been used as an effective tool to analyze such problems.Based on the development of the theory so far, we can consider the following basicquestions where each of them is interesting in its own right.

(2.2) Question. For a given pair of maps (f, g) we would like to know if itcan be deformed to a pair which is coincidence free, i.e. Coin(f ′, g′) = ∅ for some


pair (f ′, g′) homotopic to (f, g), or equivalently if the empty set belongs to thefamily of all sets Coin(f ′, g′) as (f ′, g′) runs over the set of all maps homotopic to(f, g), respectively.

The next two questions also have to do with the idea of minimizing the set ofcoincidence points,but in a more subtle situation then that of Question (2.2). Incontrast to the fixed point case, since the pair of spaces X, Y may be distinct, inprinciple this will make the study of coincidence more subtle, in particular withrespect to the understanding and formulation of the minimizing question. Let usfirst give a definition. For a set A we denote its cardinality by |A|.

(2.3) Definition. Let MC[f, g] be defined as

MC[f, g] = minf ′�f, g′�g

|{x ∈ X : f ′(x) = g′(x)}|

= minf ′�f, g′�g

{|Coin(f ′, g′)|} = MC[g, f ],

where � means homotopic.

(2.4) Question. Compute MC[f, g] by means of homotopy invariants associ-ated to the maps and the spaces involved.

The weaker question, which consists of deciding whether MC[f, g] is finite orinfinite, is already quite interesting, and constitutes a relevant step to the fullunderstanding of the coincidence problem. Of course Question (2.2) is equivalentto knowing if MC[f, g] = 0. We will see that MC[f, g] is quite often infinite. Whenthis happens, the notion of “minimizing the set of coincidence points” is not quitestraightforward, and there are alternatives formulations. Three relevant measuresof a set are:

(2.5.1) The number of its connected components.(2.5.2) The Hausdorff measure, see [HW].(2.5.3) The cohomological dimension defined below.

Let us consider case (2.5.3) and denote the Cech cohomology with coefficientsin a ring R by H( · , R). Because of the use of Poincare duality in coincidence´theory see [V], Cech cohomology is a suitable cohomology theory to measure theset Coin(f, g).

(2.6) Definition. Given a set A let the cohomological dimension of A withrespect the coefficient R, denoted by cohd(A,R), be defined by

cohd(A,R) = max{n ∈ Z : Hn(A,R) �= 0�� }.

(2.7) Question. Compute the minimum of cohd(Coin(f ′, g′), R) as f ′, g′ runsover the family of all pairs of maps homotopic to f , g, respectively.


I will leave the reader to explore other possibilities of questions concerningthe idea of minimization. We hope that the examples given below will provide abetter feeling for the above questions. Of course a specific problem may providethe reader with a very natural kind of minimizing problem.

(2.8) Remark. As a result of the development of the theory, in Section 4 wewill raise questions similar to (2.2), (2.4) and (2.7), stated in terms of the notionof Nielsen classes.

For the rest of this section we give some general results which show whenMC[f, g] is finite or infinite, and some examples of the infinite case. These re-sults indicate which are the relevant questions of coincidence theory that shouldbe analyzed in each case.

The first result explains why we consider the case dimX ≥ dimY .

(2.9) Proposition. LetX be a CW-complex and Y a manifold where dimX <dimY . Then any pair of maps f, g:X → Y can be deformed to be coincidence free.

Proof. Consider the map f × g:X → Y × Y . The pair can be deformedto be coincidence free if and only if the map f × g can be compressed to thesubspace Y ×Y −∆Y , where ∆Y is the diagonal on Y ×Y . By obstruction theory(see [Wh] or [St]) the obstructions for such a deformation lie in the cohomologygroups Hi(X, πi(Y × Y, Y × Y − ∆Y ). By either [Fad1] or [FH1], the groupsπi(Y × Y, Y × Y − ∆Y )) vanish for 0 ≤ i ≤ n − 1, where dimY = n. SincedimX < dimY it follows that all the groups vanish and the result follows. �

Now suppose that X is a finite CW-complex, Y a manifold and dimX = dimY .We show that MC[f, g] is finite in several cases.

(2.10) Proposition. Let X be a finite CW-complex and Y a manifold wheredimX = dimY . Then any pair of maps f, g:X → Y can be deformed to a pair(f ′, g′) such that Coin(f ′, g′) is finite.

Proof. Following [FH1] let us consider a cocycle which represents the obstruc-tion to deforming the pair to be coincidence free. This cocycle may be written asa sum of elementary cocycles. The local coefficient system is given by the groupring Z[π] ≈ πn(Y × Y, Y × Y −∆Y ) where π = π1(Y ). So for each n dimensionalcell, we have a map ∂en → Y × Y −∆Y which is the restriction of a map of pairs(en, ∂en) → (Y ×Y, Y ×Y −∆Y ). Now we claim that there is an extension of themap defined on the boundary to the interior, such that the number of coincidencesis finite. This follows from the identification and construction of the generators ofπn(Y × Y, Y × Y −∆Y ) ≈ Z[π1(Y )]. �

The following result also holds.


(2.11) Proposition. If X is a finite simplicial complex of dimension ≤ nand Y is a finite complex whose maximal simplices of all of dimension ≥ n, thenMC[f, g] is finite.

Proof. This result is not explicitly stated in [Sc1] but it follows immediatelyfrom [Sc, Hilfssatz 1 and Satz IIa]. �

It is natural to ask what happens to the minimal number of coincidence pointsif we deform only one of the maps. It turns out that in many cases, this minimalnumber is the same as the case where one deforms both maps. Therefore onemay take advantage of this equivalence to make some specific calculations. Moreprecisely, when the target space Y is a manifold, the number MC[f, g] may alsobe achieved by deforming only one of the maps. This is useful when studying theminimality of Coin(f ′, g′).

(2.12) Proposition. If Y is a manifold (without boundary) then the numberMC[f, g] is equal to the minimum of the cardinality of Coin(f ′, g) for all maps f ′

homotopic to f:

MC(g)[f ] = minf ′�f

|{x ∈ X : f ′(x) = g(x)}|.

Proof. Following [G2, Proposition 1.5], consider the fibred pair (Y ×Y, Y ×Y − ∆)

p2−→ Y (see [Fad1]), where p2 is the projection on the second coordi-nate. Certainly we have that MC[f, g] ≤ MC(g)[f ]. So it suffices to showthat MC(g)[f ] ≤ MC[f, g]. For this, let (f ′, g′) be an arbitrary pair. We havethat g, g′:X → Y × Y are homotopic, where g(x) = (f(x), g(x)) and g′(x) =(f ′(x), g′(x)). Let H be such a homotopy. The map g′ has a lift, namely (f ′, g′)such that (f ′, g′)(X − Coin(f ′, g′)) ⊂ Y × Y − ∆. By the lifting property offibered pairs, it follows that there is a lift H of H such that H(X−Coin(f ′, g′)) ⊂Y × Y − ∆. So H( · , 1) = (f ′′, g) and Coin(f ′′, g) ⊂ Coin(f ′, g′). Therefore#Coin(f, g) ≤ #Coin(f ′, g′) and the result follows. �

The above result was first proved in [Br1] in a stronger form.Let M,N be two compact manifolds of the same dimension with boundary, and

consider a pair of maps f, g:M → N . Coincidence theory can be studied underseveral different hypotheses on the maps f , g. The case where we assume thatone of the maps, the map g for example, is a map of pairs, namely g: (M, ∂M)→(N, ∂N), has been studied in [BSc1] and [BSc2]. In this category we define ahomotopy as a pair of homotopies ftff , gt, where ftff is a homotopy of f and gt is aboundary-preserving homotopy of g i.e. gt(∂(M)) ⊂ ∂N . Then we can define theminimal number of coincidence, also denoted by MC[f, g], in a given homotopyclass as described above. As an immediate consequence of [BSc1, Theorem 6.1]we have:


(2.13) Corollary. The minimal number of coincidence points MC[f, g] inthe category of pairs defined above is finite.

Let us present a simple example which shows that it is not possible to minimizethe number of points and the number of connected components of Coin at thesame time.

(2.14) Example. Let K be the union of the circle S1 of radius one with anarc that is one of its diameters, and let N = S1. Consider the pair of mapsf, c:K → S1 where c is the constant map at the point 1 ∈ S1 ⊂ C, and f(z) = z2

if z ∈ S1 and f(z) = 1 for z belonging to the diameter. Observe that Coin(f, c) isthe diameter, so it has one connected component. But by a small deformation off (for example by composing the map f with a small rotation of the target S1),we obtain a pair (f ′, c) such that Coin(f ′, c) consists of two points, so it is notconnected. We cannot have Coin(f1, c) = ∅ for f1 homotopic to f , since f is notnull homotopic. Also the map f1 restricted to S1 is a map of degree 2 which has aspreimage of 1 ∈ S1 at least 2 connected components. This implies that we cannotexpect to be able to minimize the number of points and the number of componentsat the same time.

The above example can be modified to the case where the codimension is posi-tive. It will have different features, and will be examined in Example (2.16).

We cannot expect MC[f, g] to be finite, even when X and Y are finite CW-complexes for which dim(X) > dim(Y ). When this happens, the two propertiesthat we must have in mind for the purpose of minimizing are the cohomologicaldimension and the number of components of Coin. In order to illustrate the non-finiteness of MC[f, g], consider the following example:

(2.15) Example. Let p:E → B be a locally-trivial fibration with connectedfibre between compact orientable manifoldsE, B withm = dim(E) > dim(B) = n,and let c:E → B be the constant map at a point b0 ∈ B. By using a Serre spectralsequence argument, we have that Hm−n(Coin(p, c),Z) = 0. Hence Coin(�� p, c) isnot finite. Now let p′ be an arbitrary map homotopic to p (not necessarily a fibremap). Let Hi( · , · ) denote the i-th Cech cohomology group. From the proof of [G5,Proposition 3.2], using the transfer as defined in [Dol, Chapter VIII, Section 10],we have that

Hm−n(p′−1(b0),Z) ∼= Hm−n(p−1(bo),Z) �= 0�� .

So MC[p, c] is not finite.

Now we consider an example where we can minimize the cohomological dimen-sion and the number of components at the same time. However, the set is not


minimal in the sense that it contains a proper subset which can be realized as theCoin of some pair.

(2.16) Example. Let X be the union of the torus Tn+2 = S1 ×S1 × Tn witha cylinder S1 × [0, 1], where we glue the boundary of the cylinder S1 × {0, 1} tothe circles S1 × {1,−1} × 1 ⊂ Tn+2 = S1 × S1 × Tn. Define f :X → Tn+1 asfollows: when restricted to Tn+1 = 1 × Tn+1, f is given by f(1, z2, . . . , zn+2) =(z2

2, . . . , zn+2) and f(z) = 1 for z in the cylinder. Observe that f is well definedand continuous. Let c:X → Tn+1 be the constant map. As in Example (2.14), wehave that Coin(f, c) is the cylinder, so it is a connected 2-dimensional manifoldwith boundary. After a small deformation (just compose f with a small rotation ofthe circle), we obtain that Coin(f ′, c) is the union of two disjoint circles. Now wewill show that Coin(f ′′, c) can be a connected complex of dimension 1 for some mapf ′′ homotopic to f . More precisely, we can make Coin(f ′′, c) to be the union of twocircles with a segment from one circle to the other. The cylinder is the quotient ofthe square under certain identifications on the boundary. The constant map fromthe square can easily be deformed, relative to the boundary, so that the preimageof the base point of the target is only the boundary. This defines the new mapf ′′ such that Coin(f ′′

1 , c) is the complex in question. Finally we will show that forany (f ′′, g′′) homotopic to (f, c), Coin(f ′′, g′′) contains two closed disjoint subsetswhose cohomological dimension is at least one. Consider the restriction of f toTn+2 = S1 × S1 × Tn, which by abuse of notation we also denote by f . This isa fibration. From the proof of Proposition (2.12), it follows that it suffices justto deform f . The image of the fundamental group f#ff : π1(Tn+2) → π1(Tn+1) hasindex 2; we consider the lift to the double covering. Let f be the lifting of f .From Example (2.15) above it follows that for any deformation of the map f , thepreimage of a point has cohomological dimension at least 1. Since the preimageof the base point by f is the same as the preimage of two points by f , the resultfollows.

(2.17) Example. Consider a pair of maps f, g:X → Y between manifolds ofdimension m, n, respectively. We claim that if there is a homology class α ∈HnHH (X,Z) such that the image of α by the composition

j∗ ◦ (f × g)∗:HnHH (X,Z) → HnHH (Y × Y,Z)→ HnHH (Y × Y, Y × Y −∆; Z)

is nontrivial, then the cohomological dimension of Coin(f ′, g′) is at least m−n forany pair f ′, g′ homotopic to f , g. From the commutative diagram below

(2.18)

HnHH (X; Z) ��

j1

��

HnHH (Y × Y ; Z)

j2

��

HiHH (X,X − Coin(f ′, g′),Z) �� HiHH (Y × Y, Y × Y −∆; Z)


the map j∗ ◦ (f ×g) factors through HnHH (X,X−Coin(f ′, g′); Z) which implies thatHnHH (X,X−Coin(f ′, g′); Z) �= 0. Using duality we obtain�� Hm−n(Coin(f ′, g′),Z) �= 0��and the result follows.

Given homeomorphisms φ:X → X and ψ: Y → Y , for each map f :X → Y wemay define a new map ψ ◦ f ◦ (φ)−1. Let Homeo(X), Homeo(Y ) denote the groupof the homeomorphisms of X and Y respectively, under the operation h∗k = h◦k.The map Homeo(X) × Homeo(Y )× Y X → Y X defined above is an action of thegroup Homeo(X) × Homeo(Y ), with the product group structure on the functionspace Y X . This action factors through an action on the set of homotopy classesof maps [X, Y ]. It is straightforward to see that the coincidence set Coin(f, g) ishomeomorphic to the Coin(ψ ◦ f ◦ (φ)−1, ψ ◦ g ◦ (φ)−1)) of the two new maps. Animmediate consequence of this fact is that for all the coincidence questions thathave been treated so far, the answer for a pair (f, g) is the same as that for thepair (ψ ◦ f ◦ (φ)−1, ψ ◦ g ◦ (ψ)−1). Therefore for practical purposes if we want toverify if a property is true for all pairs, it suffices to do so for a set of homotopyclasses [f, g] which contains one representative in each orbit with respect to theaction of Hom(X)×Hom(Y ) on the set of homotopy classes of pair of maps. Thisfact can be useful depending on how much knowledge one has about the orbits.In the case of maps between surfaces the naive idea above has been used withsuccess in at least two situations. The first was in [GZ], [BGZ2] to study Coinof pairs of maps between surfaces where the second map is the constant map (orthe root case). The second was in [GJi] in order to study bounds of the index oncoincidence Nielsen classes.

(2.19) Remark. It is common to say that fixed point theory is a particularcase of coincidence theory, it being enough to consider the particular case wherethe second map f2ff = id. This is in general correct but some concepts definedin coincidence theory when applied to the situation where the second map is theidentity, do not always correspond to the concept defined in fixed point theory.See for example [BGZ1] for a discussion about the number MC (see also [BSc1])defined above and [GKe] for the notion of Wecken homotopies.

3. The Lefschetz–Hopf trace formula

This section is devoted to finding an efficient and computable way to decidewhether a pair of maps cannot be deformed to be coincidence free. This is Ques-tion (2.2). The great achievement for this problem was a formula given by Lefschetzin the period 1923–1927, see the series of papers [L1]–[L4]. For a given pair of mapsf, g:M → N where M,N are closed orientable manifolds of the same dimension,the Lefschetz formula associates an integer to such a pair. Despite the fact that


it would have been more natural in those days to find such a formula to detectfixed points, the Lefschetz formula detects coincidence. This is a formula whichclearly depends only on the homotopy classes of the maps f and g. Immediatelyafter Lefschetz found the formula, Hopf extended it to pairs f, id:K → K whereid is the identity (the fixed point case) and K is a finite complex, not necessarilya manifold. So the formula nowadays is called the Lefschetz–Hopf trace formula.In more modern language, we start by describing the results obtained relative tothis formula for closed orientable manifolds. Then we move on to discuss furtherresults for other categories of spaces. The most recent result that we will treathere is the case of pairs of maps f, g:M → N , where both manifolds are compactwith nonempty boundary, g(∂M) ⊂ ∂N and one of the maps is orientation true(see below for the definition of orientation true).

Let M,N be orientable closed manifolds and f, g:M → N a pair of maps. (Formore details see [V, Chapter VI].)

(3.1) Definition. Let the homomorphism ϑi:Hi(N,Q) → Hi(N,Q) be thecompositionD−1

N ◦(f)∗◦DM◦(g)∗, where D is the Poincare duality homomorphism.´The Lefschetz number L(f, g) is defined by the Lefschetz–Hopf trace formula

L(f, g) =n∑

i=1

(−1)itrace(ϑi).

The Lefschetz number can also be defined in cohomological terms, allowing theuse of the multiplicative structure of cohomology and of cohomology operations inthe computation of the Lefschetz number, see [Fad2], [GO].

(3.2) Definition (See [V, Chapter VI]). Let the homomorphism

ϑi:HiHH (M,Q) → HiHH (M,Q)

be the composite DM ◦ (g)∗ ◦D−1N ◦ (f)∗ , where DM :Hn−i(M,Q) → HiHH (M,Q) is

the Poincare duality homomorphism. The´ Lefschetz number L(f, g) is defined bythe Lefschetz–Hopf trace formula

L(f, g) =n∑

i=1

(−1)itrace(ϑi).

The equalities trace(ϑi) = trace(ϑn−i), 1 ≤ i ≤ n, imply L(f, g) = (−1)nL(f, g)for the two Lefschetz numbers defined above. The Lefschetz number is anti-commutative: L(f, g) = (−1)nL(g, f).

The importance of this number is due to the fact that its non-vanishing guaran-tees the existence of a coincidence point. The number L(f, g) is homotopy invari-ant; hence the statements about the existence of a coincidence can be reformulatedin the following stronger form. More precisely:


(3.3) Theorem. Let f, g:M → N be two continuous maps between closedorientable manifolds of the same dimension n. If L(f, g) �= 0�� then Coin(f ′, g′) �=�� ∅for any f ′, g′ homotopic to f, g, respectively.

We will see that the development described above, will be repeated in severalother categories. They have some similarities. At the end we show how a theoremof the type of Theorem (3.3) above is proved.

In 1980, Nakaoka (see [Na]), studied coincidences of fibre-preserving maps, asubject that we do not treat here. In this work he developed a type of Lefschetz–Hopf formula for fibre-preserving maps, where the fibres are manifolds. However,the results there are also stated for the case where the fibres are compact ori-entable manifolds with boundary (see [Na, Section 8]). So a Lefschetz–Hopf traceformula for manifolds with boundary appears for the first time in [Na]. Laterin 1992, Mukherjea ([Mu]) independently studied the Lefschetz–Hopf trace for-mula for manifolds with boundary in more detail. We now describe this case.

Consider the category of compact orientable manifolds with boundary, andlet M , N be two such manifolds of the same dimension. Let f, g:M → N bea pair of maps such that the second map satisfies the property g(∂M) ⊂ ∂N .By a homotopy of the pair we mean a pair of homotopies (H,G), where H is anarbitrary homotopy of f and G is a boundary-preserving homotopy of g i.e. Gt isa boundary-preserving map for all t ∈ [0, 1].

Let DM :H∗(M, ∂M ; Q)→HnHH −∗(M,Q) and DN :H∗(N, ∂N ; Q)→HnHH −∗(N,Q)be the Poincare–Lefschetz duality homomorphisms.´

(3.4) Definition. Given a pair of maps f, g:M → N , where g(∂M) ⊂ ∂N , letthe homomorphism ϑi:Hi(N, ∂N ; Q) → Hi(N, ∂N ; Q) be the composition D−1

N ◦(f)∗ ◦DM ◦ (g)∗. The Lefschetz–Hopf trace formula is given by:

L(f, g) =n∑

i=1

(−1)itrace(ϑi).

Then we have the following result.

(3.5) Theorem. Let f, g:M → N be two continuous maps between compactorientable manifolds of the same dimension n and with boundary. If the Lefschetz–Hopf number above satisfies L(f, g) �= 0�� then Coin(f ′, g′) �=�� ∅ for any f ′, g′ ho-motopic to f, g, respectively, where the homotopy between g and g′ is boundarypreserving.

Let us exploit this result a little. The Brouwer Fixed Point Theorem says thatevery continuous map f :Dn → Dn (Dn is the n-dimensional disk) has a fixedpoint. Nevertheless a pair of maps f, id:Dn → Dn, where the second map is


the identity, can be deformed as a pair such that Coin(f ′, id′) = ∅. In fact, thisis the case for any pair of maps f, g:Dn → Dn, because any map is homotopicto an arbitrary constant map. But the suitable generalization of the fixed pointquestion would be: given any deformation (f ′, id′) of the pair (f, id) where thehomotopy connecting id to id′ preserves the boundary Sn−1 of Dn, is Coin(f ′, id′)non empty? From [BrSc1] we find that not only is the answer yes in this case,but there are many other maps g: (Dn, ∂Dn) → (Dn, ∂Dn) which have a similarproperty to the identity in the above example. In more detail:

(3.6) Definition. A map g: (M, ∂M) → (N, ∂N) is said to be coincidenceproducing if for every map f :M → N we have Coin(f, g) �=�� ∅.

(3.7) Proposition. The identity id: (Dn, ∂Dn) → (Dn, ∂Dn) is coincidenceproducing. Further, any map id′ homotopic to id as a map of pairs is coincidenceproducing.

Proof. Let us compute L(f, id). Since the pair (Dn, ∂Dn) only has coho-mology in dimension n, it follows that the induced homomorphism D−1

N ◦ (f)∗ ◦DM ◦ (g)∗:Hn(Dn, ∂Dn) → (Dn, ∂Dn) is the identity because f∗ff :H0HH (Dn,Q) →H0HH (Dn,Q) is the identity independent of the map f . So the result follows. �

It would be nice to know if the property of being a coincidence producing mapis a property of the homotopy class. See [BwSc1, Section 7].

So far, all the results concern orientable manifolds. In 1997, see [GJ], the Lef-schetz–Hopf formula for nonorientable manifolds was treated. The developmentof the theory in general in this case is similar. But there are differences in thedetails; we shall make this explicit and give more information. Following [GJ], foran arbitrary manifold M we can define a local orientation along any path, andalong a closed path, the orientation is either preserved or reversed. Homotopicclosed paths have the same behavior and so this yields a homomorphism from thefundamental group to Z2. Denote by ΓW be the local system over a manifoldW ,where the coefficients are the rationals and the action is given by the orientationbehavior of the loop. This local system is called the orientation system withrational coefficients of the manifold. It follows that ΓW is trivial if and only if Wis orientable.

Let us recall a concept which was formally defined by P. Olum in [O]. A mapf :M → N is called orientation true if for every element α ∈ π1(M), both α andf#ff (α) have the same sign, i.e. either both loops are orientation preserving or bothare orientation reversing. If f :M → N is orientation true then the induced localsystem f∗(ΓN ) is given by

α · 1 = f#ff (α) · 1 = sign(f#ff (α)) · 1 = sign(α) · 1,


where 1 is the unit of Q. Hence f∗(ΓN) = ΓM , and f induces a map

f∗:H∗(N,ΓN) → H∗(M,ΓM )

on the cohomology with local coefficients.We start with the case of compact manifolds without boundary. Let

DM :H∗(M,ΓM ) → HnHH −∗(M,Q) and DN :H∗(N,ΓN ) → HnHH −∗(N,Q)

be the Poincare duality homomorphisms with local coefficients.´We will consider the situation where the second map g is orientation true. Then

we define:

(3.8) Definition. For maps f, g:M → N where g is orientation true, let thehomomorphism ϑi:Hi(N,Q) → Hi(N,Q) be the composite D−1

N ◦(g)∗◦DM ◦(f)∗.The Lefschetz–Hopf trace formula is given by

L(f, g) =n∑

i=1

(−1)itrace(ϑi).

Let us observe that in the case where both maps are orientation true we cancompute L(f, g) and L(g, f). Recall that in the orientable case we showed af-ter Definition (3.2) that L(f, g) = (−1)nL(g, f) where n is the dimension of themanifold. This is not the same in the nonorientable case and the examples belowillustrate this fact.

(3.9) Example. Let fkff :S2 → S2 be an odd map (fkff (−x) = −fkff (x)) of degreek (k must then be an odd number). This map induces a map fkff :RP 2 → RP 2 andwe have deg(fkff ) = deg(fkff ) = k. On the other hand, fkff induces an isomorphismof fundamental groups, hence is orientation true. Let us fix two odd integers k, land compute the Lefschetz number L(fkff , flff ) (with rational coefficients Q). SinceRP 2 is Q-acyclic, only the sequence

H0(RP 2; Q)f∗

k−→ H0(RP 2; Q) −→ H2(RP 2; Γ)f∗

l−→ H2(RP 2; Γ) −→ H0(RP 2; Q)

may make a non-zero contribution to L(fkff , flff ). Thus L(fkff , flff ) = deg(flff ) = l. Buton the other hand, L(flff , fkff ) = deg(fkff ) = k.

(3.10) Example. The following example shows that L(f, g) �= 0 does not imply��L(g, f) �= 0. Let�� p, c:S2 → RP 2, where p is the projection and c is the constantmap. It is easy to see that L(p, c) = 0 but L(c, p) = 2.

This type of example can even be given for selfmaps.


(3.11) Example. Let K be the Klein bottle. Take f, g:K → K where theinduced maps on the fundamental group are given by f#ff (α) = 1, f#ff (β) = β,g#(α) = α and g#(β) = β3. We have that L(f, g) = 0 and L(g, f) = 2.

Now we move to the case with boundary. Let f, g:M → N , where g is ori-entation true. We have two essentially distinct geometric situations, and foreach case a formula may be defined and be used to detect coincidences. Thefirst one is if g: (M, ∂M) → (N, ∂N) is a map of pairs. The second one is iff : (M, ∂M) → (N, ∂N) is a map of pairs.

Let us consider the first case.

(3.12) Definition. For maps f, g:M → N and g: (M, ∂M) → (N, ∂N) amap of pairs, where g is orientation true, let the homomorphism ϑi:Hi(N ; Q) →Hi(N ; Q) be the composite D−1

N ◦ (g)∗ ◦ DM ◦ (f)∗. The Lefschetz–Hopf traceformula is given by

L′(f, g) =n∑

i=1

(−1)itrace(ϑi).

Then we have:

(3.13) Theorem. Let f, g:M → N be two continuous maps between closedmanifolds of the same dimension n. If the Lefschetz–Hopf number above satisfiesL′(f, g) �= 0�� then Coin(f ′, g′) �=�� ∅ for any f ′, g′ homotopic to f, g, respectively,where the homotopy between g and g′ is boundary preserving.

Now let us consider the second case: g is orientation true and f(∂M) ⊂ ∂N .

(3.14) Definition. For maps f, g:M → N and f : (M, ∂M) → (N, ∂N) a mapof pairs, where g is orientation true, let the homomorphism ϑi:Hi(N, ∂N ; Q) →Hi(N, ∂N ; Q) be the composite D−1

N ◦ (g)∗ ◦DM ◦ (f)∗. The Lefschetz–Hopf traceformula is given by

L′′(f, g) =n∑

i=1

(−1)itrace(ϑi).

Then we have:

(3.15) Theorem. Let f, g:M → N be two continuous maps between closedmanifolds of the same dimension n. If the Lefschetz–Hopf number above satisfiesL′′(f, g) �= 0�� then Coin(f ′, g′) �=�� ∅ for any f ′, g′ homotopic to f, g, respectively,where the homotopy between g and g′ is boundary preserving.

It will immediately occur to the reader that if the two maps f, g are orientationtrue, we have two Lefschetz numbers, and both may be used to detect coincidencepoints in the homotopy class of the pair (f, g). What is the relation between these


two numbers? From [GJ, Theorem 4.9] let f |∂M , g|∂M be the restrictions of f , g,respectively, to the boundary ofM . By hypothesis it turns out that they are mapsf |∂M , g|∂M : ∂M → ∂N where ∂M, ∂N are closed manifolds of dimension n − 1.Since ∂g is also orientation true, the Lefschetz number is defined and we have:

(3.16) Proposition. L′(f, g)− L′′(f, g) = L(f |∂M , g|∂M).

We have seen above that under several different hypotheses, there is a theoremon the existence of coincidence for a given pair of maps. In all cases the proof ofsuch theorems follows the same pattern. So we will illustrate here the case whereM , N are manifolds with boundary and f, g:M → N , where g is orientation trueand boundary-preserving.

From a Thom class in Hn(intN × intN, intN × intN −∆; Q× ΓN ), one candefine (see [GJ, Section 3]) a cohomology class

U ′NU ∈ Hn(intN × (N, ∂N); Q× ΓM).

We define the index of the pair f, g as the image of the fundamental class zM ∈HnHH (M, ∂M ; ΓM) under the sequence of homomorphisms:

HnHH (M, ∂M ; ΓM )d′

∗−→ HnHH (M × (M, ∂M); Q× ΓM )

f×g)∗−−−−−−− →−− HnHH (intN × (N, ∂N); Q× ΓN )〈U ′

N , · 〉−−−−−−− →−− Q

and denote ind′(f, g) = 〈U ′NU , (f × g)∗d′

∗(zM )〉 (see more about indices in the nextsection).

Here is the main result which is also called the normalization property.

(3.17) Theorem (Normalization). Let f, g:M → N be a pair of maps betweenclosed n-manifolds with g orientation true and g(∂M) ⊂ ∂N . Then

L′(f, g) = ind′(f, g).

The proof of the above result, after all the preparation concerning the Thomclass, is similar to the proof of the corresponding result in [V, Chapter 6]. Animmediate consequence of the above theorem is the existence of a coincidencein the suitable homotopy class of the pair whenever the number given by theLefschetz–Hopf trace formula is not zero.

(3.18) Corollary. If the pair (f, g) can be deformed to be coincidence freethen L(f, g) is zero.

Proof. By properties of homology and cohomology we have the equality

〈U ′NU , (f × g)∗d

′∗(zM )〉 = 〈d′∗ ◦ (f × g)∗U ′

NU , (zM )〉.


If the pair (f, g) can be deformed to be coincidence free, then we have a factoriza-tion h:M → N ×N −∆ of (f, g), and (f ×g)∗U ′

NU can be computed as (h)∗ ◦ i∗U ′NU

which is zero since U ′NU is the restriction of a cohomology class of the pair. The

result follows. �

The results of this section lead one to pose the question about the necessityof the condition on the Lefschetz–Hopf number, that is, about the validity of theconverse to the above assertion. A great part of contemporary coincidence theorydeals with results which are, in some sense, converses to this homotopical formof the Lefschetz–Hopf theorems [Fu], [Fad2], [Fad3]. One also seeks a sharperestimate of the minimal number of coincidence points. This is the purpose of thenext two sections.

4. Nielsen and Reidemeister coincidence classes

In this section we define Nielsen and Reidemeister coincidence classes. Thereis a natural way to associate a Reidemeister class to each non-empty Nielsen classof a pair of maps (f, g). The correspondence is injective and suggests that theReidemeister classes can be regarded as a coordinate system to index the Nielsenclasses. We define equivalence relations on the set of Nielsen classes and on the setof Reidemeister class. Two such classes are said to be (H,G)-related where (H,G)is a self-homotopy of the pair (f, g).

4.1. Nielsen coincidence classes.

(4.1) Definition. Let f, g:X → Y be a pair of continuous mappings. Twocoincidence points x0, x1 ∈ Coin(f, g) are Nielsen equivalent if there exists a pathλ in X such that λ(0) = x0, λ(1) = x1 and f1(λ) is homotopic to f2ff (λ) relative tothe end points. An equivalence class is called a Nielsen class.

Under very mild conditions, one can show that a Nielsen class is a closed andopen subset of Coin(f, g). For instance suppose that X is locally path connected,and Y is semi-locally simply connected.

(4.2) Proposition. Each Nielsen class is open in Coin(f, g).

Proof. Let x1 ∈ Coin(f, g). It follows from the hypotheses that there existsa neighbourhood W of f(x1) = g(x1) = y1 in Y such that any loop in W withbase point y1 is homotopically trivial. Consider a path-connected neighbourhoodU ⊂ f−1(W ) ∩ g−1(W ) of x1. Now take x2 ∈ U ∩ Coin(f, g) and λ a path in Uconnecting x1 to x2. Then f ◦ λ and g ◦ λ are two paths in W with the same endpoints. Applying the property of W , it follows that the two paths are homotopicrelative to the end points. Therefore the two points belong to the same Nielsenclass and the result follows. �


If we further assume that Y is Hausdorff we can show that a Nielsen class isclosed.

(4.3) Proposition. Under the above hypothesis, a Nielsen class of (f, g) isclosed in Coin(f, g).

Proof. Since Y is Hausdorff it follows that the diagonal ∆Y ⊂ Y ×Y is closed.But Coin(f, g) = (f×g)−1(∆Y ), where f×g:X → Y ×Y . It follows that Coin(f, g)is closed. So it suffices to show that one Nielsen class is closed in X. Let x0 be anaccumulation point of a Nielsen class F . By the previous proof there is an openset such that all the coincidence points in the neighbourhood belong to the sameNielsen class of x0, and the result follows. �

If X and Y satisfy all of the above hypotheses then the previous two resultsshow:

(4.4) Corollary. The set Coin(f, g) is a closed subset of X, and each Nielsenclass is open and closed in Coin(f, g).

The challenge is to define or associate to a Nielsen class an algebraic objectwhich at the same time is computable and which has the property that if thisalgebraic object is not zero, than we must have a coincidence point. We explorethis aspect in Section 5.

4.2. Reidemeister coincidence classes. Reidemeister classes are one of thebasic tools in the study of coincidence theory. They can be regarded as a coor-dinate system to index the Nielsen classes. Motivated by the study of maps onnonorientable manifolds, we give a definition of Reidemeister classes which in thisspecial case will distinguish some of them. Otherwise it will coincide with theusual one.

Let f, g:X → Y be a pair of maps between arbitrary spaces and let x0 ∈ Xsatisfy f(x0) = g(x0). Then we define:

(4.5) Definition. The Reidemeister classes of the pair (f, g) are the set ofequivalence classes of elements of π1(Y, y0) given by the relation α ≡ β if and onlyif there exists θ ∈ π1(X, x0) such that β = g#(θ)α(f#ff (θ))−1. For maps betweenmanifolds, a class α ∈ π1(Y ) is called a defective class if there exist θ such thatα = g#(θ)α(f#ff (θ))−1 and sign(θ)sign((f#ff (θ)) = −1.

Suppose that Coin(f, g) is not empty, x0 is a coincidence point and y0 = f(x0) =g(x0) is its image. We will define an injection from the set of Nielsen classes intothe set R(f, g) of Reidemeister classes. For x1 ∈ Coin(f, g), let λ be any path fromx0 to x1. The path g(λ)∗f(λ)−1 is a loop and it defines an element of π1(Y, y0). Itis straightforward to see that different paths from x0 to x1 define elements of the


same Reidemeister class, and Nielsen equivalent coincidence points are associatedwith elements of the same Reidemeister class.

4.3. Related classes. Let (f, g) be a pair of maps. Given a Nielsen class,one would like to know if this class can disappear under homotopy. In order toformalize this idea, we define a notion of related classes. Let (H,G) be a homotopyof the pairs (f, g) and (f1, g1).

(4.6) Definition. Two Nielsen classes C1 ∈ Coin(f, g) and C2 ∈ Coin(f1, g1)are said to be (H,G)-related if there is a path λ starting at a point of C1 andending at a point of C2 such that H(λ(t), t) is end point homotopic to G(λ(t), t).

For the case of two classes of the same pair of maps we define:

(4.7) Definition. Two classes C1, C2 ∈ Coin(f, g) are said to be homotopyrelated if there is a pair of self-homotopies (H,G) of (f, g) such that C1 is (H,G)-related to C2.

A similar notion can be defined for Reidemeister classes.

(4.8) Definition. Given a self-homotopy (H,G) of the pair (f, g), considerthe pair of loops (w1, w2) ∈ π1(Y ) × π1(Y ), where w1(t) = G(x0, t) and w2(t) =H(x0, t). We say that [α] is (H,G)-related to [β] if [w2αw

−11 ] = [β]. They are

homotopy related if they are (H,G)-related for some pair of self homotopies.

We have a relation on Nielsen classes and another on Reidemeister classes. Themap defined above from the set of Nielsen classes into the set of Reidemeisterclasses preserves these relations.

(4.9) Proposition. The correspondence which associates a Nielsen coinci-dence class to a Reidemeister class preserves related classes.

Proof. Let Coin(f, g) be nonempty and let x0 be a base point. Considertwo coincidence points x1, x2 ∈ Coin(f, g), and suppose that they are related.So there exists a self homotopy (F,G) and a path λ from x1 to x2 such thatH(λ(t), t) is end point homotopic to G(λ(t), t). Take α to be any path connectingx0 to x1. In the space X× I, the path λ2 = (x0, (1− t)) ∗ (α, 0) ∗ (λ(t), t) connects(x0, 1) to (x2, 1), and is end homotopic to the path λ′

2 = (α, 1) ∗ (β, 1). The loopG( · , 1)(λ′

2)∗F ( · , 1)(λ′2)−1) defines an element of the Reidemeister class associated

to the element x2 and this element is homotopic to

G( · , 1)(λ2) ∗ F ( · , 1)(λ2)−1)

= G((x0, (1− t)) ∗ (α, 0) ∗ (λ(t), t))F ((λ(t), t)−1 ∗ (α, 0)−1 ∗ (x0, (1− t))−1.

Therefore it follows that β = w2αw−11 and the result follows. �

Now we may define a la Brooks [Br2] a “geometric essential Nielsen coincidence`class”.


(4.10) Definition. A Nielsen coincidence class is said to be geometric essentialif it is not (H,G)-related to the empty class. Otherwise it is called geometricinessential. Therefore we can define the geometric Nielsen coincidence number tobe the number of geometric essential coincidence classes.

4.4. Jiang subgroup for coincidences. In 1964, Jiang [Ji1] defined a sub-group J(f) ⊂ π1(X) for every map f :X → X, which was later called the Jiangsubgroup of the map f . This group was very useful for fixed point theory. In 1965,Gottlieb (see [Gt]) defined and studied the subgroup of the fundamental groupcorresponding to the above group, for the case where f is the identity map. Thisgroup was defined in terms of function spaces. This subgroup, denoted by G1(X),was later called the first Gottlieb group of X. The first Gottlieb group not onlyhas a close relationship with fixed point theory, but also with other branches ofmathematics, like the theory of fibrations, rational homotopy theory and groupactions (see [Op]).

We adapt the above notion of Jiang subgroup to the coincidence case. Let ZX

be the space of functions from X to Z with the compact-open topology, where Xis a space with base point x0. Then we have the evaluation map e:ZX → Y givenby e(g) = g(x0).

(4.11) Definition. The Jiang subgroup J(f), corresponding to a map f :X→Z is the image

e#(π1(Y X , f)) ⊂ π1(Z, f(x0)).

We apply the above definition to the coincidence case. Namely, given a pair ofmaps f, g:X → Y , consider the map f × g:X → Y × Y . So we define two groupsrelated to the pair (f, g).

(4.12) Definition. The subgroup J(f × g) is called the Jiang subgroup ofthe pair (f, g). Also we denote by JcJJ (f, g) ⊂ π1(Y ) the smallest subgroup whichcontains the image of J(f × g) under the map π1(Y ) × π1(Y ) → π1(Y ) given by(α, β)→ β(α)−1.

In the next section we show some applications of the Jiang subgroup. We define:

(4.13) Definition. A pair (X, Y ) is called a Jiang pair for coincidences ifJcJJ (f, g) = π1(Y ) for all pairs of maps.

Whenever JcJJ (f, g) = π1(Y ) we have nice properties for the coincidence theoryof the pair (f, g). Assuming that X, Y satisfies some mild conditions such as X iscompact and that Nielsen classes are closed and open, as a direct consequence ofthe definitions above we have:


(4.14) Theorem. If JcJJ (f, g) = π1(Y ) then

(4.14.1) All Reidemeister classes are related. Hence all Nielsen classes are re-lated.

(4.14.2) Either N(f, g) = 0 or all Nielsen classes are essential.(4.14.3) If the geometric Nielsen number is not zero, then it is finite and is equal

to the number of Reidemeister classes (which thus has to be finite).

Proof. Part (4.14.1). It suffices to show that the class of the trivial element1 ∈ π1(Y ) is related to the class of an arbitrary element α ∈ π1(Y ). By hypothesisthere exist θ1 × θ2 ∈ J(f) × J(g) such that θ1θ−1

2 = α. This implies that theclasses [1], [α] are (H,G)-related for H,G self-homotopies corresponding to θ1, θ2,respectively.

Part (4.14.2) follows directly from part (4.14.1).Part (4.14.3). Since X is compact and the Nielsen classes are open and closed

it follows that the number of Nielsen classes is finite. Since by hypothesis thereexists one Nielsen class which is essential, then there is a Reidemeister which isessential. It follows from (4.14.1) that all the other Nielsen classes corresponds toessential Reidemeister classes and conversely. So the result follows. �

We leave to the reader to explore the relation between Jiang spaces and Jiangpairs of spaces. Of course if Y is a Jiang space (for example a Lie group or anH-space) then (X, Y ) is a Jiang pair for any space X.

To conclude this section we will show how Reidemeister classes arise naturallyin the context of obstruction theory to deform a pair f, g:K → N , where N is an-dimensional manifold and K is a CW-complex. For a set J , denote the sum ofZ indexed by the set J by Z[J ].

(4.15) Proposition. The group in which the primary obstruction to makingf, g coincidence free lies is the direct sum Hn(K; Z[Ri]) where Ri runs over allReidemeister classes. In particular, if K is also an n-dimensional manifold thenthis group is isomorphic to the direct sum of A′s where A is either Z or Z2 (notnecessarily the same) indexed by the set of Reidemeister classes.

The main idea behind this Proposition comes from [FH1]. But for further detailsand in a more explicit form see [G4].

5. Index of an isolated subset of the coincidence set

We start this section by making a few comments about index theory for fixedpoints. Such an index has a long history dating back to Hopf [H1]. It has been fur-ther developed and defined for a quite general family of spaces, including the cate-gories of finite polyhedra and compact metric ANR’s. For more details see [BGZ1].


In coincidence theory the situation is more subtle, and we will make more restric-tions on the spaces involved when compared with the fixed point case. Considera pair of maps f, g:X → Y and an isolated subset F ⊂ Coin(f, g). In some situa-tions, we will be able to define the coincidence index of the subset F . This will beequivalent to the so-called local coincidence index under certain conditions.

5.1. Maps between orientable manifolds. In coincidence theory for closedorientable manifoldsM1MM and M2MM of the same dimension, a local index of a subsetF was defined. Consider homology with integral coefficient Z. Following [V,Chapter VI] we define the local coincidence index as follows.

(5.1) Definition. Let M1MM ,M2MM be n-dimensional closed orientable manifolds,V an open subset of M1MM , and f, g: V → M2MM mappings such that C = Coin(f, g)is compact. The coincidence index i(V ; f, g) of the pair (f, g) defined on C is theinteger given by the image of the fundamental class of M1MM under the composition

HnHH (M1MM ) → HnHH (M1MM ,M1MM \W )→ HnHH (V, V \W )

→ HnHH (M2MM ×M2MM ,M2MM ×M2MM −∆(M2MM )) ∼= Z,

where the second map is the excision, the third is induced by (f,g)(x)=(f(x),g(x)),and W satisfies Coin(f, g) ⊂W ⊂W ⊂ V .

Because of the following property (5.2.1) we can drop the reference to V andwrite i(Coin(f, g)) instead of i(V ; f, g).

(5.2) Properties. The index satisfies the following properties.

(5.2.1) (Localization) i(V ; f, g) = i(V ′; f |V ′ , g|V ′) for every open set V ′ ⊂ Vcontaining C. This allows us to write i(Coin(f, g)) = i(V ; f, g).

(5.2.2) (Additivity) If V is a finite union of open sets ViVV , i = 1, . . . , r, and C isdisjoint union of compact sets CiCC with CiCC ⊂ ViVV , then

i(V ; f, g) = i(V1VV , f1, g1) + . . .+ i(VrVV ; frff , gr) where (fjff , gj) = (f |VjVV , g|VjVV ).

(5.2.3) (Homotopy invariance) If ftff , gt: V →M2MM , 0 ≤ t ≤ 1, are homotopies suchthat K = {x ∈ V : there exists t ∈ [0, 1] with ftff (x) = gt(x)} is compact,then i(Coin(f0ff , g0)) = i(Coin(f1, g1)).

(5.2.4) (Multiplicativity) Let f, g: V → M2MM , f ′, g′: V ′ → M ′2MM be maps. Then

Coin(f×f ′, g×g′) = Coin(f, g)×Coin(f ′, g′) and i(Coin(f×f ′, g×g′)) =i(Coin(f, g)) · i(Coin(f ′, g′)).

(5.2.5) (Normalization) For a pair f, g:M1MM → M2MM of globally defined maps theindex i(Coin(f, g)) is equal to the Lefschetz number L(f, g) of the pair(f, g).

Let us point out that the commutativity property, which holds for fixed pointtheory, does not hold for coincidence theory.

Documents

HANDBOOK OF TOPOLOGICAL FIXED P OINT THEORY · 5. Periodic points andbraid theory T. Matsuoka 171 6. Fixed point theory of multivalued weighted maps J. Pejsachowicz, R. Skiba 217