Math. Ann. 238, 123--129 (1978) © by Springer-Verlag 1978

Normalizers of Modular Groups*

Morris Newman

Institute for the Interdisciplinary Applications of Algebra and Combinatorics and Department of Mathematics, University of California, Santa Barbara, CA 93106, USA

Introduction

Let F=SL( t ,Z ) , s'2=SL(t,R), t an integer >1. Let M~(Z) denote the ring of integral t x t matrices. If G is a subgroup of F, Nr(G ) will denote the normalizer of G in F, and No(G ) the normalizer of G in g2. In the case of the classical modular group PSL(2,Z) the quotient group No(G)/G provides information about the group of conformal self mappings of the Riemann surface ~o associated with G\j/F, ~g the upper half-plane, a subject studied by Lehner and the author for the special case of the congruence groups Fo(n ) in [1]. Here we wish to examine these normalizers in more generality. A known result is that if G is of finite index in F and if the genus of 5 e is >2, then N~(G)/G is finite. One of the purposes of this paper is to generalize this result. Another is to derive explicit bounds for the order of N~(G)/G. We also make some comments about normal subgroups of the classical modular group PSL(2, Z).

For any positive integer n, let A(n) denote the normal closure in F of (I + El z)" = I + nEl z. Then A (n) contains I + nE~, 1 < a, fl < t, ~ # fl. Here E~p is the matrix with a 1 in position (e, fl), and 0 elsewhere. It is known by the work of Mennicke ([2]) that if t > 3, then A (n) is a subgroup of F of finite index, and that any noncentral normal subgroup of F is of finite index. In fact A(n)=F(n), the principal congruence subgroup of level n. If t = 2 , however, this only holds for n < 5. For n > 6, A (n) is of infinite index in F.

Let iv, denote the fully invariant subgroup of F generated by the n m powers of the elements of F. Then F"3A(n), and Mennicke's result implies that F" is a subgroup of F of finite index for t > 3. For t = 2, there are values of n for which F" is of infinite index in F, although the recent work on the Burnside problem is required to establish this fact.

Let A be any matrix of Mr(Z). Then there are matrices U, V of F such that

UA V= S = diag(s 1, sz,..., s,, 0 ..... 0).

* This work was supported by NSF grant MCS 76-82923

0025-5831/78/0238/0123/$01.40

124 M. Newman

Here r is the rank of A, and s 1, s 2 ..... s r are nonzero integers (known as the invariant .[actors of A), such that s i divides si+l, 1 <_i<_r-1. Furthermore, the invariant factors are unique, up to sign. s t is just the greatest common divisor of the entries of A, and if A is nonsingular, sls2...s t is the determinant of A. S is said to be in Smith Normal Form. Similarly, it is known that there is a matrix W of F such that WA (or A W) is either upper triangular or lower triangular, as desired.

The Principal Lemma

The results derived in this paper depend on the following lemma:

Lemma 1. Let M be an element oJ ~2 such that

M-1A(n)MCF.

1 Then M=-oA, where A is an integral matrix such that s a = l , si[n, 2<=i<t, and

d = det(A) = 0 t. Thus did- 1.

Pro@ Let e, fl be integers such that l < e , t int , ~+fl. Since I+nE,~eA(N), M - 1 (I + nE~#) M ~ F, so that riM- 1 E ~ M e M t (Z). Put

M ~- [mi j ] , M - 1 _~. [Mi j - ] .

Since E,# = [~i,6#~], we have

E~t3M= [k=~ 61,6pkmk~l =['i,mt3~],

M-1E,#M=[k~=Mik6k,m#j ] =[Mi,m,j].

Thus

nM~,mtj j is an integer. (1)

First choose c t= l . Then there is an i= i (1) such that Mm),~:~0, since M is nonsingular. Then (1) implies that

mcj=Ole#),fl:t: 1, 1 <j < t, (2)

where 01 is independent of fl and j, and c#j is an integer. Next choose ~t = 2. Then there is an i= i (2) such that M~t2),2 4=0. Then (1) implies that

m#i = Ozdoj , fl :# 2, 1 < j < t, (3)

where 0 2 is independent of fl and j and d¢~ is an integer. Now (2) and (3) taken together imply that

M=DtAx =DEA2, where A1,A2eMt(Z), (4)

and

D 1 = diag(zl , 01, 01 .... ,0a), D2 = diag(02, %, 02,..., 02).

Normalizers of Modular Groups 125

It follows that the entries of D~ 1D 2 =A1A ~- ~ are rational.

Suppose first that t>2 . Then - - , zl 01 zl 02 ~-2 are rational, which implies that ~-1 is

1 rational. This easily implies that M has the form ~A, where A~Mt(Z ).

Next suppose that t = 2. Then (4) certainly implies that

-I0 Choose U ~ F so that

so that eflad= 1. We have

nM- 1E 12 M e M 2(Z),

so that

nU- 1M- 1ElzMU~M~(Z ).

A brief calculation now shows that

nfl - d is integral, ~ a

_ 1 which implies that fl~ is rational. As before, this implies that M has the form ~ A,

where A~M2(Z ). Thus M always has the form

1 M = -(gA, A~Mt(Z) ,

and we may clearly assume as well that s 1 = s I (A)= 1. Now write UAV=S, U, V~F, S=diag(s~,s 2 ..... st) in Smith Normal Form.

Then certainly

M -~ {U- l(I+nE~t~) U} M~F,

V- 1M-1 {U- 1(1+nEap) U} M V ~ F ,

S ~ 1 (I + nE~) Se F,

nS- l E~S~Mt(Z)"

Choose ~ = t, fl = 1. Then we find that

n ~ = n is an integer, st st

so that stln. But then s~In, 2 <= iN t, since sits r Finally, Or= det(A), since det(M)= 1, and det(A)=slSa...stln t- 1. This completes the proof.

126 M. Newman

Consequences oJ the Lemma. We first prove a maximality property of the group F with respect to D.

Theorem L Let G be a group such that f23G3F, (G:F) finite. Then G=F.

Prooj. Since (G :Y) is finite, there is a positive integer n such that G"CF. It follows that if M is any element of G and A any element of F, then M - 1A"M = ( M - IAM)"~G"CF. Thus

M - 1A(n)MCM- IG"MCG~CF.

1 Hence M must have the form ~ A, where AeMt(Z ) (we do not require the other

properties possessed by A which are guaranteed by Lemma 1, in this proofl. Choose U, V e F so that UAV is in Smith Normal Form. Then UMV=D is a real diagonal matrix of determinant 1 which belongs to G. Furthermore, D"eF. This implies that the diagonal elements of D" are all _+ 1, which in turn implies that the diagonal elements olD are all _+ 1. Hence D~F, and therefore so does M. It follows that G = F, and the proof is complete.

The next result concerns normalizers.

Theorem 2. Let G be a subgroup oJ F such that G3A(n). Then (No(G):Nr(G)) is finite.

Proof Let M~Na(G). Then M-1GM=G, so that

M-1A(n) M C M - I G M = G C F .

Lemma 1 now implies that

1 M = ~ A, A t M,(Z), det(A)[n t- 1.

Consider the totality S~ of elements M = 1A of Na(G) with det(A)-- _+d, d>0 . If

M~, M2eS d and are left equivalent then M 2 = U M 1, where U~F. Since U = M 2 M ~ ~, U also belongs to No(G). Hence UeFc~Na(G)=Nr(G ). It follows that M~ and M 2 belong to the same right coset of N~(G) modulo Nr(G ). Hence the number of elements of S~ which are inequivalent modulo Nr(G) cannot exceed the number of left equivalence classes of matrices of determinant _+ d [the Hermite Normal Form class number, denoted by H(d)] and this number is finite (see [3], pp. 15-21). It follows that (N~(G):Nr(G)) is finite, and that

(Na(G):Nr(G))<= ~, H(d). d[n t - 1

This completes the proof. The exact value of H(d) is known ([3]). In fact,

p k + j _ _ 1 u(d)= I]

vklla j= 1 p / - 1

When t = 2, H(d) becomes a(d), the sum of the divisors of d.

Normalizers of Modular Groups 127

An immediate corollary is

Theorem 3. Let G be a subgroup oJ F oj finite index. Then N~(G)/G is a finite group.

Proof Since (F:G) is finite, G must contain d(n) for some positive integer n. We have

(N~(G) : G) = (N~(G) :Nr(G) ) (Nr(G):G) .

Now (No(G):Nr(G)) is finite by Theorem 2, and (Nr(G):G) is finite, since G is a subgroup o f f of finite index. Hence (Ne(G):G) is finite and the conclusion follows.

We also have a result which we state as a lemma.

Lemma 2. Let G be a normal subgroup oJ F which contains A (n). Then Ne(G)= F.

Proof Since G is a normal subgroup of F, Nr(G)= F. By Theorem 2, (N~(G):F) is finite. By Theorem 1, Ne(G)= F. This completes the proof.

This result raises the following question: Can the condition that G contains A(n) be eliminated? For t > 3, Mennicke's work supplies an affirmative answer. The result in this case becomes that Ne(G) = F, provided that G is any non-central normal subgroup, and so of finite index. The situation is not so simple in the case of the classical modular group. It is possible to have normal subgroups in this case containing no parabolic elements at all. For example, we have

Theorem 4. Let n be an integer > 1. Then the commutator subgroup F(n)' oJ the principal congruence subgroup F(n) oJ F = PSL(2, Z) contains no parabolic elements (and so must be a subgroup oJ F oJ infinite index),

Prooj. It is known that F(n) is a free group of finite rank, and that S"= [~ n] m a y l

be taken as one of its free generators. Thus if W is any element of F(n)', the exponent sum of S" in W is 0. This implies that no nonzero power of S" lies in F(n)', so that no conjugate of such a power lies in F(n)', since F(n)' is a fully invariant subgroup of the normal subgroup F(n), and hence a normal subgroup of F. Hence

F(n)' cannotcontainanyconjugateofanypowerof[10 l]=I+E12, andtheproof

is concluded.

is nown o,eme t is

rati°nally similar t° [ 0-t a+dl ]. This justifies asking whether or not a nontrivial

normal subgroup of SL(2, Z) must contain A, for some positive integer n, where A,

standsforthenormalclosurein F=PSL(2,Z) of[O_l lnl. Unfortunately, this is

not the case. In fact, we have

Theorem 5. The order oj the group G=F/A, is either 1, 2, 3, or 6.

128

Pro@ Let x stand for [_O1 10], Y for [ _ ~

x, y with defining relations

XZ = y 3 = 1, X =(xy) n.

Thus (xy) 2" = x 2 = 1. Also, writing the last relation of (5) as

x = x (yx)"- i y ,

we get

y - 1 = (yx)"- 1,

1 = ( y - 1) 3 = ( y x ) 3"-3 ,

which implies that

( xyp" - 3 = 1.

This and the fact that ( x y ) 2" = 1 imply that

(xy) 6 = 1.

M. Newman

_ 11]. Then G is the group generated by

(5)

The remainder of the argument goes by cases, depending on the residue class of n modulo 6. It will be sufficient to summarize the results, since the computations are routine.

nmodulo 6 IG[ A, 0 3 F 3 1 2 r 2

2 1 F (6) 3 6 F' 4 1 F 5 2 F 2

Having made these negative remarks, we now prove the desired generalization of Lemma 2. The technique used is to go over to the integral group ring.

Theorem 6. Let G be a non-central normal subgroup oJ F. Then Na(G)= F.

Prooy. As noted previously, it is only necessary to consider the case F = SL(2, Z). If G contains a parabolic element, then G contains A (n) for some positive integer n and the conclusion follows from Lemma 2. If G does not contain a parabolic

element, then G must contain an element C = [: dbl such that bc #0. We now note

the following identity:

It follows that if M is any element of No(G) and U any element of F, then M - ~ U - 1B U M has integral entries, so that M - a U - 1 ( I - 2cE 1 :) U M belongs to F.

Normalizers of Modular Groups 129

1 Just as in the proof of Lemma 1, this implies that M must have the form M = ~ A,

where A is an integral matrix. Hence matrices U, V~ F may be found such that

D=UMVisareal diagonal matrix of determinant 1, say D = d i a g (~,~). Then D \ /

also belongs to Ne(G). Hence D v belongs to Ne(G) for all integral p, and so

DPAD-P=[ ac_zp c ~t2db]~F. (7)

Suppose now that ~:t: +__ 1. Then either Is{ < 1, or lc~t > 1. If I~t < 1, choose p so large that tct2pbl < 1. If }at > 1, choose p so large that la- 2pcl < 1. In either case, we contradict (7), since bc+O. It follows that a = + t, which implies that D = ___I, so that M = _ U -1 V - I ~ F . Hence N a ( G ) = F , and the proof is concluded.

References

L Lehner, J., Newman, M. : Weierstrass points of Fo(n). Ann. of Math. 79, 360-368 (1964) 2. Mennicke, J.L. : Finite factor groups of the unimodular group. Ann. of Math. 81, 31-37 (1965) 3. Newman,M. : Integral matrices. New York: Academic Press 1972

Received April 11, 1978