OSCILLATORY INTEGRAL OPERATORS

IN SCATTERING THEORY

Andrew Comech

Mathematics, Columbia UniversityNew York, NY 10027

Abstract. We consider a particular Fourier integral operator with foldingcanonical relations, which arises in scattering theory: the Radon Transform ofMelrose and Taylor. We obtain the regularity properties of this operator when

the obstacle admits tangent planes with contact of precise order k (Theorem1.1 and its Corollary).

For these purposes, we derive asymptotic estimates for oscillatory integral

operators in Rn with folding canonical relations (Theorem 2.2). Asymptoticscorrespond to vanishing principal curvature of a fold of one of the projec-

tions from the canonical relation, and to small support of the localization ofoscillatory integral operator.

1. Introduction and results

From the fundamental paper of R.B. Melrose and M.E. Taylor [1985] weknow that important characteristics of scattering, such as corrections to theKirchhoff approximation and the behavior of the scattering amplitude forsmall angles of scattering, depend on the properties of the following Radon

1991 Mathematics Subject Classification. Primary 43A32; Secondary 35L05, 35S30.

Transform:

(1.1)

R : E ′(R× Sn)→ D′(R×B),

Ru(t, r) =∫

R×Sn

δ(t− s− 〈r, ω〉)u(s, ω) ds dΩ.

Here r is a vector in Rn+1, pointing to the boundary B of a scatterer K ⊂Rn+1. For the scattering in three dimensions we put n = 2. We alwaysassume thatK is a compact region with the boundary being a smooth strictlyconvex hypersurface. A unit vector ω ∈ Sn corresponds to a direction of lightrays. The variables t and s refer to the moments of time.

We will call (1.1) the Radon Transform of Melrose and Taylor.We would like to know the regularity properties of R:

(1.2) R : Hs(R× Sn)→ Hs+n2−µ(R×B).

If R were a non-degenerate Fourier integral operator, the value of µ in(1.2) would be 0. Instead, according to R.B. Melrose and M.E. Taylor, R is aFourier integral operator with degenerate canonical relation. Their argumentin [1985] works for the case when the hypersurface B is strictly convex andits Gaussian curvature is strictly positive. In this case, the operator R canbe reduced to the normal form. Then one can show that there is a loss ofµ = 1/6 derivatives in (1.2).

The analogous result for Fourier integral operators with folding canonicalrelations is proved by L. Hormander [1985]. His proof was also based on thereduction to the normal form.

Due to the results of A. Greenleaf and A. Seeger [1994], there is a loss ofat most µ = 1/4 derivatives in (1.2).

What is the optimal value of µ if the curvature vanishes of some finiteorder? In this paper, we will develop a technique which will allow us toanswer this question in a particular situation. Exactly, we will be able toconsider the case when B admits a tangent plane with the contact of preciseorder k (in the sense specified after the theorem and its corollary):

Theorem 1.1. Let B be a smooth hypersurface in Rn+1, and let its Gaussiancurvature vanish at an isolated point ro. If the tangent plane at ro has acontact of precise order k > 1 with B, then the Radon Transform R localizednear ro extends to a continuous operator

Hs(R)⊗ L2(Sn) R−→ Hs+n2−

12

k2k+1 (R)⊗ L2loc(B), for any real s.

Due to the particular form of the Radon Transform (1.1), the smoothingwith respect to time-variables implies the same smoothing effect with respectto other variables. This results in the following continuity ofR in the Sobolevspaces:

Corollary 1.2. Let K be a convex compact domain in Rn+1 with a smoothboundary B = ∂K. Let the Gaussian curvature of B vanishes only at iso-lated points. If at these points the boundary B admits tangent planes withcontact of precise order not greater than k, then the operator R extends to acontinuous operator

Hs(R× Sn) R−→ Hs+n2−

12

k2k+1 (R×B), for any real s.

Contact of precise order. Let the hypersurface B in Rn+1 be locally givenby the graph xn+1 = b (x1, . . . , xn) of some smooth function b ∈ C∞(Rn),defined in the neighborhood of the origin in Rn. We assume that b(0) = 0,∇xb(0) = 0.

Definition 1.3 (Contact of precise order k).We say that the hyperplane xn+1 = 0 has a contact of precise order k

with the graph xn+1 = b(x) of the function b ∈ C∞(Rn), if b(0) = 0,∇xb(0) = 0, and if the Hessian [∂xi∂xj b] is isotropic and vanishes uniformlyof order k − 1. That is, for any unit vectors u, v ∈ Rn,

(1.3)

uiuj∂2b (x)

∂xi∂xj≥ c ‖x‖k−1 ,

∣

∣

∣

∣

uivj∂2b (x)

∂xi∂xj

∣

∣

∣

∣

≤ C ‖x‖k−1 ,

where ‖x‖2 ≡ x21 + · · ·+ x2n, 0 < c < C <∞.

The first condition defines the contact of order at most k, while the secondcondition defines the contact of order at least k. Of course, the conditions(1.3) do not depend on the choice of Euclidean coordinates (x1, . . . , xn).

As an example, we can take a function b (x) = f(‖x‖), where f ∈ C∞(R)

is such that crk−1 ≤ f ′′(r) ≤ Crk−1; say, b(x) = ‖x‖k+1, for odd k.Let us specify the required smoothness of b(x). In the argument we use,

we only need b ∈ C2n+2(Rn). Hence, in the above example, k could also beany real number greater than or equal to 2n+ 1.

Notice that if the Gaussian curvature of B does not vanish, then k = 1,and formally the theorem gives the loss of µ = 1

2k

2k+1 = 1/6 derivatives. If,

instead, k becomes large, then µ approaches 1/4. This gives an interpolationbetween the results of Melrose – Taylor and Greenleaf – Seeger.

Relation to oscillatory integral operators. The Radon Transform (1.1) canbe written as a Fourier integral operator:

(1.4) Ru(t, x) =∫

R×R×Sn

e−iλ(t−s−S(x,ϑ)) ψ(x, ϑ)u(s, ϑ)dλ

2πds dnϑ,

here x and ϑ are local coordinates on B and on Sn, and the phase shift

(1.5) S(x, ϑ) = 〈r(x), ω(ϑ)〉

is a dot product of n + 1-dimensional vectors pointing to x ∈ B and ϑ ∈Sn. We introduced some smooth cut-off function ψ(x, ϑ), which localizesthe integral operator to the region where the local coordinates x and ϑ aredefined.

As it was pointed out by R.B. Melrose and M.E. Taylor, the canonicalrelation associated with (1.4) has folding singularities. We will analyze thesesingularities in more detail after reducing the problem to oscillatory integraloperators.

Following the paper of D.H. Phong and E.M. Stein [1991], we rewrite (1.4)as a pseudo-differential operator in one dimension, with the “operator-valuedsymbol”:

Ru(t, x) =∫

R

dλ

2πe−iλt

∫

Rn

dnϑ eiλS(x,ϑ) ψ(x, ϑ)

∫

R

ds eiλsu(s, ϑ).

The symbol a(λ) of this pseudo-differential operator acts on the Fourier trans-form of u(s, ϑ) (with respect to s) as

(1.6) a(λ) u(λ) =

∫

Rn

dnϑ eiλS(x,ϑ) ψ(x, ϑ) u(λ, ϑ).

The standard methods of the theory of ΨDO show that the smoothingeffect of the operator R (with respect to the time-variables) coincides withthe rate of decay of the operator norm of a(λ). Precisely,

(1.7)

if ‖a(λ)‖L2comp(Rn)→L2loc(Rn) ≤ const λ−

n2+µ,

then Hs(R)⊗ L2comp(Rn)R−→ Hs+n

2−µ(R)⊗ L2loc(Rn).

Now we proceed to studying the estimates for oscillatory integral opera-tors, like (1.6).

In Section 2, we discuss oscillatory integral operators with degeneratecanonical relations and formulate our main technical result (Theorem 2.2).The idea of the proof is in Section 3, and the details are in Sections 4 and 5.

We approach the particular oscillatory integral operator with the phasefunction (1.5) in Section 6.

2. Oscillatory integral operators

with folding canonical relations

In this section, we formulate the asymptotic estimates for oscillatory in-tegral operators with folding canonical relations.

We study oscillatory integral operators Tλ , given by

(2.1)Tλ : u(ϑ) 7→ Tλu(x) =

∫

RnR

eiλS(x,ϑ) ψ(x, ϑ)u(ϑ) dϑ,

x ∈ RnL, ϑ ∈ Rn

R,

where ψ is a smooth compactly supported function. We want to know therate of high-frequency decay of the operator norm ‖Tλ‖L2→L2 for large λ.The simplest example is the Fourier transform: S(x, ϑ) = x · ϑ. Then this iseasy to show that ‖Tλ‖ ≤ const λ−

n2 .

The classical result by L. Hormander [1971] is the estimate for the casewhen the mixed Hessian H = Sxϑ =

[

∂xi∂ϑjS]

, i, j = 1, . . . , n, is non-

degenerate: detH 6= 0. Then Tλ is bounded from L2 to L2, with the norm

(2.2) ‖Tλ‖L2→L2 ≤ const λ−n2 .

Let us consider the diagram

(2.3)

RnL× Rn

R

ı−→ C → T ∗RnL× T ∗Rn

R

πL π

R

T ∗RnL

T ∗RnR

where ı : (x, ϑ) 7→ (x, Sx)×(ϑ,−Sϑ). The image C = ı(RnL×Rn

R) is the canon-

ical relation associated with (2.1). The maps πL

and πRare the projections

from C onto the first and second factors of T ∗RnL× T ∗Rn

R.

The maps πL= π

L ı and π

R= π

R ı from Rn

L× Rn

Rfail to be diffeomor-

phisms on the critical variety

(2.4) L ≡ (x, ϑ) | detH(x, ϑ) = 0 ,

where we thus have Ker dπL6= 0 and Ker dπ

R6= 0.

This all means that the estimate (2.2) is valid when both projections in(2.3) are local diffeomorphisms.

We say that πLis a fold if the kernel of dπ

Lis one-dimensional and transver-

sal to the critical variety L. Moreover, πLis a Whitney fold if the determinant

of the mixed Hessian vanishes only of order one on L:

(2.5) dϑ detH |L6= 0.

From now on, we always assume that πLis a Whitney fold.

Since dimKer dπL= 1, the rank of the mixed Hessian H on the critical

variety L is equal to n − 1. Therefore, the dimension of Ker dπR

is also 1.Still, the projection π

Ris not necessarily a fold; in general, the differential

dx detH can vanish at some points of the critical variety L.According to R.B. Melrose and M.E. Taylor (see also Y.B. Pan and C.D.

Sogge [1990] and S. Cuccagna [1995]), if both projections from the associatedcanonical relation C are at most Whitney folds, then the decay of the normof Tλ is given by

(2.6) ‖Tλ‖L2→L2 ≤ const λ−n2+

16 .

The estimate due to A. Greenleaf and A. Seeger [1994] holds when at leastone of the projections is a Whitney fold (we always assume it is π

L); then

the norm of Tλ decays as

(2.7) ‖Tλ‖L2→L2 ≤ const λ−n2+

14 .

Let us see what canonical relation corresponds to the phase functionS(x, ϑ) defined in (1.5). The cotangent space T ∗rB can be identified (viathe standard scalar product) with the tangent plane at the same point r.This shows that the differential dx〈r(x), ω(ϑ)〉 is represented by the orthog-onal projection of ω(ϑ) onto Tr(x)B. Therefore, the singular part of the map

πL: (x, ϑ) 7→ (x, dx〈r(x), ω(ϑ)〉)

corresponds to the orthogonal projection from Sn onto the hyperplane TrB.This projection has a Whitney fold at the points ω such that 〈nr, ω〉 = 0,here nr is the unit normal to B at the point r. We conclude that both dπ

L

and dπRare degenerate on the critical variety

(2.8) L = (x, ϑ) | 〈nr(x), ω(ϑ)〉 = 0.

Intuitively, if TrB and TωSn are not orthogonal (〈nr, ω〉 6= 0), then, afterproper localization, the oscillatory integral operator is basically the Fouriertransform and has no critical points.

Let us consider the other projection from C. The singular part of the map

πR: x, ϑ 7→ ϑ, −dϑ〈r(x), ω(ϑ)〉

corresponds to the orthogonal projection from B onto Tω(ϑ)Sn. As long as Bis strictly convex, π

Ris a fold. It is also a Whitney fold at the points r ∈ B

where the principal curvature in the direction ω is different from zero.All this suggests that we consider an asymptotic situation, when π

Ris

a Whitney fold, but its principal curvature is asymptotically small. Letus explain what we call a principal curvature of a Whitney fold: locally,a singular part of a Whitney fold corresponds to the projection from theparabola y = 1

2σx2 onto the y-axis; then, roughly, σ is the principal curvature

of this fold.Let us try to translate this into analytical language, for the operator

(2.1) with a general phase function S(x, ϑ). According to (2.5), we willassume that near the critical variety L = detH = 0 we have the bounds‖∇ϑ detH‖ ≥ const and ‖∇x detH‖ ≥ σ. If σ becomes small, the behaviorof the projection π

Rbecomes more degenerate, and we expect that there is

a certain asymptotic behavior for ‖Tλ‖, corresponding to small values of σ.This behavior turns out to be

‖Tλ‖ ∼ λ−n2+

16σ−

16 .

In Theorem 2.2, we will derive this asymptote together with asymptoticscorresponding to small support of the localization of Tλ near some point(xo, ϑo):

Tλδε ≡ ρ((x− xo)/δ) Tλ %((ϑ− ϑo)/ε).

Pseudoconvex maps. We need to require that πR

in (2.3) be convex, in acertain sense. First, for an N ×N -matrix A, we define the bound from belowfor its action:

(2.9) min(A) ≡ inf‖u‖=1

‖Au‖ .

If A is non-degenerate, then min(A) =∥

∥A−1∥

∥

−1.

There is the following bound for min(A):

min(A) ≥ m(A) ≡ |detA|‖A‖N−1End(RN )

.

To see this, it suffices to treat A as a linear endomorphism of CN , so thatdetA would be a product of the eigenvalues.

Definition 2.1 (c-pseudoconvexity).

Let U and V be two regions in RN . Given c > 0, the map Φ : U → V iscalled c-pseudoconvex on U , if on any connected subset O ⊂ U it satisfies thecondition

(2.10) ‖Φ(y)− Φ(x)‖ ≥ c ‖y − x‖ infz∈O

min(∇Φ(z)).

Here ∇Φ(x) is the Jacoby matrix corresponding to the change of variablesx 7→ Φ(x). The value of min(∗) at a given point is defined in (2.9).

Note that any function f ∈ C1(R) is then pseudoconvex with c = 1.

The typical example when (2.10) does not hold is a map Φ which is a localdiffeomorphism but not a bijection. Still, if U is compact and min(∇Φ) ≥const , then for any c < 1 there is a finite partition U = ∪Uj , such that Φ isc-pseudoconvex on each Uj .

Condition (2.10) is non-trivial if ∇Φ is degenerate at some points of U .

Asymptotic estimates for oscillatory integral operators. Now we formulateour main result about oscillatory integral operators. We consider an operator(2.1) localized near some point (xo, ϑo):

(2.11) Tλδεu(x) =

∫

Rn

ρ((x− xo)/δ) %((ϑ− ϑo)/ε) eiλS(x,ϑ) ψ(x, ϑ)u(ϑ) dϑ,

here ρ(x) and %(ϑ) are smooth functions supported in a unit ball centeredin the origin; δ and ε are two small parameters (less than 1). The integralkernel of Tλδε is thus supported in Bn

δ (xo)× Bnε (ϑo), where Bn

r (p) is a ball inRn of radius r, centered at p. For simplicity, we assume that (xo, ϑo) is shiftedto the origin in Rn

L× Rn

R. We will use the notation Dδε ≡ Bn

δ (0)× Bnε (0).

In the Theorem we want to formulate, both πLand π

Rare assumed to be

Whitney folds. The local coordinates x = (x′, xn) and ϑ = (ϑ′, ϑn) are chosenso that the vectors ∂ϑn and ∂xn are transversal to L = (x, ϑ) | detH = 0,while H

′ = ∇x′∇ϑ′S is a non-degenerate matrix.

Theorem 2.2 (Asymptotic estimates for oscillatory integral operators).

Let S(x, ϑ) ∈ C2n+3(Dδε) and ψ(x, ϑ) ∈ C2n+1(Dδε).

Let the mixed Hessian H = Sxϑ be of rank at least n − 1, with its non-degenerate part given by H

′ = ∇x′∇ϑ′S, |detH′| ≥ const > 0.

If detH vanishes of the first order in xn and ϑn, so that

(2.12) |∂ϑn detH| ≥ const > 0 and σ ≡ infDδε

|∂xn detH| > 0,

the map πR|ϑ: x 7→ Sϑ(x, ϑ) satisfies the pseudoconvexity condition (2.10)

with some c ≥ const > 0, and if the following inequalities hold:

supDδε

(

‖H′−1‖ ‖∇x′∂ϑnS‖)

supDδε

‖∇ϑ′ detH‖

infDδε

|∂ϑn detH| ≤1

2,(2.13)

supDδε

(

‖H′−1‖ ‖∇ϑ′∂xnS‖)

supDδε

‖∇x′ detH‖

infDδε

|∂xn detH| ≤1

2,(2.14)

then there are the following estimates on the norm of Tλδε in (2.11):

‖Tλδε‖ ≤ const λ−n−12 δ

12 ε

12 ,(2.15)

‖Tλδε‖ ≤ const λ−n2+

14 δ

14 ,(2.16)

‖Tλδε‖ ≤ const λ−n2+

16σ−

16 ,(2.17)

‖Tλδε‖ ≤ const λ−n2+

14σ−

14 ε

14 .(2.18)

The constants in (2.15)-(2.18) do not depend on λ, δ, ε, and σ.

Let us say a couple of words about the conditions entering Theorem 2.2,and about the estimates stated there.

The conditions (2.12) simply say that πLis a Whitney fold and that π

Ris

a degenerating Whitney fold (if σ is asymptotically small).The left-hand sides of both (2.13) and (2.14) are zeroes if we assume that

S(x, ϑ) splits into purely non-degenerate and degenerate parts, S(x, ϑ) =S1(x

′, ϑ′) + S2(xn, ϑn). The problem then reduces to one-dimensional oscil-latory integral operators with folding canonical relations, when everything ismuch simpler. The geometrical meaning of the condition (2.13) is that thekernel of dπ

Lis within a certain cone about the direction of ∇ϑ detH. The

condition (2.14) requires the same of Ker πRand ∇x detH. We will need these

conditions when proving the almost orthogonality relations (Section 5).The requirement that the map x 7→ Sϑ is convex is in fact guaranteed by

(2.14); intuitively, “any fold is convex”. Not to overload the paper, we onlygive a simple geometric argument which proves the pseudoconvexity of themap π

R|ϑfor the phase function (1.5) (see Section 6).

The estimate (2.15) on Tλδε is straightforward. The factor λ−n−12 is L.

Hormander’s estimate (2.2) in the dimension n− 1, since H has the rank notless than n−1. Such estimates are considered by Y.B. Pan [1991]. The factor

δ12 ε

12 appears due to the size of the support of the integral kernel of Tλδε in

the variables xn and ϑn. No assumptions on the behavior of πLand π

Rare

needed.The estimate (2.16) holds when π

Lis a Whitney fold, and actually no

assumption on πRis needed. This is a counterpart of the estimate (2.7) due

to A. Greenleaf and A. Seeger.Other estimates on the norm of Tλδε hold when π

Ris also a Whitney

fold. The estimate (2.17) is a counterpart of (2.6), and it gives an asymptotecorresponding to a small principal curvature of the Whitney fold π

R. The

last estimate (2.18) is weak and “does not have a classical analog”. Roughly,this estimate follows from (2.16), since the size of x-support in the criticaldirection is controlled by the ratio ε/σ.

3. Idea of proof of Theorem 2.2

We follow the papers of D.H. Phong and E.M. Stein [1991], [1994], andD.H. Phong [1994], where the estimates for oscillatory integral operators inone dimension were considered.

We use the dyadic decomposition with respect to a distance from thecritical variety detH = 0. Take a function β ∈ C∞(R), such that β(t) = 1for |t| ≤ 1, β(t) = 0 for |t| ≥ 2. We define β(t) ∈ C∞o ([1/2, 2]) by β(t) =β(t) − β(2t) for t ≥ 0, β(t) = 0 for t ≤ 0. We also define β±(t) = β(±t).Then we have the following partition of 1:

1 = β(2N0t) +∑

±

∑

N<N0

β±(2N t), t ∈ R.

We will apply this partition to the operator Tλδε, decomposing it with respectto detH(x, ϑ) (“the distance from the critical variety”). We define:

(3.1) T u(x) =

∫

ρ(x/δ) %(ϑ/ε) eiλS(x,ϑ) ψ(x, ϑ) β(−1 detH)u(ϑ) dϑ,

(3.2) T ±u(x) =

∫

ρ(x/δ) %(ϑ/ε) eiλS(x,ϑ) ψ(x, ϑ)β±(−1 detH)u(ϑ) dϑ,

here = 2−N , N ∈ Z.We rewrite the operator (2.11) as a finite sum of operators (3.1) and (3.2),

(3.3) Tλδε = T o +∑

±

≤Λ∑

>o

T ±, = 2−N , N ∈ Z,

where o = 2−N0 is chosen up to our convenience; Λ = 2N1 ≥ sup |detH|.The crucial property of T

± is that on the support of its integral kernel thedeterminant of the mixed Hessian is non-degenerate: |detH| ∼ (precisely,|detH| ≥ /2, in accordance with the Uncertainty Principle). Then we cantry to use a variant of L. Hormander’s estimate (2.2). Of course, this estimatebecomes worse for smaller values of (see (3.14) below).

Later we will not write ±-indexes for the operators T ±.

The operator T o is responsible for a tiny neighborhood of the criticalvariety, of size o. We expect that its norm decreases as we take smallervalues of o (see (3.13) below).

We will obtain the estimates on∥

∥T ∥

∥ and∥

∥T ∥

∥ using a spatial localizationand then the almost orthogonality argument. We follow the preprint of S.Cuccagna [1995], who used a similar localization in his analytic proof of theL2-estimates on the oscillatory integral operators with Whitney folds. Theidea of almost orthogonality argument (the Cotlar-Stein Lemma) can be readfrom the book of E.M. Stein [1993].

We need a spatial partition of 1:

(3.4) 1 =∑

n∈Z

χ(t− n), t ∈ R, suppχ ⊂ [−1, 1].

We take two sets of integers, X = X ′, Xn = Xi ∈ Zn and Θ ∈ Zn, andlocalize the integral kernels of T and T , multiplying them by

χ(Σ−1x′ −X ′) ≡n−1∏

i′=1

χ(Σ−1xi′ −Xi′), χ(σ−1xn −Xn),

and

χ(−1ϑ−Θ) ≡n∏

i=1

χ(−1ϑi −Θi),

where

(3.5) σ ≡ infDδε

|∂xn detH| , Σ ≡ supDδε

‖∇x detH‖ , Σ ≥ σ > 0.

We decompose the operators (3.1) and (3.2):

(3.6) T =∑

X∈Zn,Θ∈Zn

T XΘ, T =

∑

X∈Zn,Θ∈Zn

T XΘ,

where T XΘ has the following integral kernel:

(3.7)

K(T XΘ)(x, ϑ) = ρ(x/δ) %(ϑ/ε) eiλS(x,ϑ) ψ(x, ϑ)β(−1 detH(x, ϑ))

× χ(Σ−1x′ −X ′)χ(σ−1xn −Xn)χ(−1ϑ−Θ).

The integral kernel of T XΘ is given by the same expression, with the func-

tion β instead of β.Referring to sizes of x- and ϑ- support of the integral kernels of T

XΘ andT XΘ, we will use the notations

(3.8) ∆σ ≡ min

δ,σ

, ∆Σ ≡ min

δ,Σ

≤ ∆σ, E ≡ minε, .

The estimates on the norms of T and T are due to the following lemma:

Lemma 3.1. There are the following estimates on T XΘ and T

XΘ:∥

∥T XΘ

∥

∥

2 ≤ const λ−n+1∆σ E,(3.9)∥

∥T XΘ

∥

∥

2 ≤ const λ−n−1.(3.10)

As long as is large enough, so that the right-hand side of (3.10) is notlarger than that of (3.9),

(3.11) −1 ≤ λ∆σ E,

the operators T XΘ and T

XΘ satisfy the almost orthogonality relations.

Precisely,∥

∥T XΘ T

YW

∗∥

∥ ,∥

∥T XΘ∗T

YW

∗∥

∥ ,∥

∥T XΘ T

YW

∗∥

∥ , and∥

∥T XΘ∗ T

YW

∥

∥ are allbounded by

(3.12) const λ−n−1 γ2(Y −X, W −Θ).

Here γ is a function on Zn × Zn such that∑

X∈Zn,Θ∈Zn

|γ(X,Θ)| ≤ const .

We will prove this lemma in Section 4 (the estimates (3.9) and (3.10)) andin Section 5 (almost orthogonality relations).

The application of the Cotlar-Stein Lemma proves the following corollary:

Corollary 3.2. There are the following estimates on T and T ±:

∥

∥T ∥

∥ ≤ const λ−n−12

(

min

δ,σ

minε, )12

,(3.13)

∥

∥T ±

∥

∥ ≤ const λ−n2 −

12 .(3.14)

We take o in (3.3) such that the bounds (3.13) and (3.14) coincide:

(3.15) −1o = λmin

δ,oσ

minε, o ≡ λ∆σ E.

Then the estimate ‖Tλδε‖ ≤ const λ−n2 −

12

o gives all the estimates (2.15)-(2.18) stated in Theorem 2.2. (¤)

For example, to obtain the estimate (2.17), we take −1o = λ · oσ · o.

4. Proof of Lemma 3.1: individual estimates

We are going to derive the individual estimates (3.9) and (3.10) stated inLemma 3.1.

In the argument, we usually omit the function ψ, and occasionally otherfunctions when they are irrelevant for the argument. We keep in mind thattheir derivatives arise each time when we integrate by parts, and thereforethe resulting estimates depend on the bounds on these functions and on theirderivatives (up to some finite order).

Estimate∥

∥T XΘ

∥

∥

2 ≤ const λ−n+1∆σ E.

For the convenience, we will write τ for the operator T XΘ:

τu(x) =

∫

χ(Σ−1x′ −X ′)χ(σ−1xn −Xn) ρ(x/δ) %(ϑ/ε)

× eiλS(x,ϑ) ψ(x, ϑ) β(−1 detH)χ(−1ϑ−Θ)u(ϑ) dϑ.

First of all, we note that for the operator τxnϑn : E ′(Rn−1) → D′(Rn−1)with the integral kernel given by K(τ)(x, ϑ) with xn, ϑn fixed, there is theestimate

(4.1) ‖τxnϑn‖L2(Rn−1)→L2(Rn−1) ≤ const λ−n−12 ,

since |det (∇x′∇ϑ′S)| = |detH′| ≥ const > 0, and the estimate (2.2) for

oscillatory integral operators with non-degenerate canonical relations applies.This estimate is uniform in xn and ϑn. It is also uniform in δ, ε, and .

The size of the integral kernel of τ in the variable ϑn is minε, ≡ E.Therefore,

|τu(x)|2 =∣

∣

∣

∣

∫

dϑn τxnϑn [u(∗, ϑn)] (x′)∣

∣

∣

∣

2

≤ E∫

dϑn |τxnϑn [u(∗, ϑn)] (x′)|2 ,

here we have written τxnϑn [u(∗, ϑn)] (x′) =∫

K(τ)(x, ϑ)u(ϑ′, ϑn) dn−1ϑ′.

Now let us integrate |τu(x)|2 with respect to x, recalling that the size ofthe support of the integral kernel of τ in xn is min

δ, σ

= ∆σ:

∫

dx |τu(x)|2 ≤ ∆σ

∫

dx′E

∫

dϑn |τxnϑn [u(∗, ϑn)] (x′)|2

≤∫

dx |τu(x)|2 ≤ const λ−(n−1)∆σ E

∫

dnϑ |u(ϑ)|2 .

We conclude that ‖τ‖ ≤ const λ−n−12 ∆

12σE

12 . This proves the estimate (3.9)

of Lemma 3.1.

This estimate is analogous to (2.15), since the support of the operator τin the variables xn and ϑn is bounded by ∆σ and E, respectively.

Estimate∥

∥T XΘ

∥

∥

2 ≤ const λ−n−1. For this estimate, it is crucial that

the integral kernel of T XΘ is localized away from the critical variety, so that

|detH| ≥ /2.We will write τ for T

XΘ:

τu(x) =

∫

χ(Σ−1x′ −X ′)χ(σ−1xn −Xn) ρ(x/δ) %(ϑ/ε)

× eiλS(x,ϑ) ψ(x, ϑ)β(−1 detH)χ(−1ϑ−Θ)u(ϑ) dϑ.

Let us consider the integral kernel of ττ ∗:

(4.2) K(ττ∗)(x, y) =

∫

dnϑ eiλ(S(x,ϑ)−S(y,ϑ)) × . . . .

We insert into (4.2) the operator

Lϑ =1

iλ· (Sϑ(x, ϑ)− Sϑ(y, ϑ)) · ∇ϑ

‖Sϑ(x, ϑ)− Sϑ(y, ϑ)‖2,

which is the identity when acting on the exponential, and integrate by parts.When acting on cut-offs, ∇ϑ gives a factor bounded by max

ε−1, −1

≡E−1. The derivative ∇ϑ can act on the denominator of Lϑ itself, also givinga factor bounded by const −1:

‖Sxϑ(x, ϑ)− Sxϑ(y, ϑ)‖‖Sϑ(x, ϑ)− Sϑ(y, ϑ)‖

≤ const ‖x− y‖ ‖x− y‖ ≤ const

−1.

The bound ‖Sϑ(x, ϑ)− Sϑ(y, ϑ)‖ ≥ const ‖x− y‖ which we have usedis guaranteed by the assumption that the map π

R|ϑis pseudoconvex. (If

the support of β(−1 detH) is not simply connected, we need a separatetreatment for each simply connected part.)

Integration by parts n+1 times is thus equivalent to adding the followingfactor to the integral kernel (4.2) of ττ ∗:

(4.3) const1

1 + |λE ‖Sϑ(x, ϑ)− Sϑ(y, ϑ)‖|n+1.

To apply the Schur lemma, we need to integrate the absolute value of(4.2), with the extra factor (4.3), with respect to x. To do so, we integrate

with respect to η = Sϑ(x, ϑ) and divide by the determinant of the Jacobymatrix. The Jacoby matrix is given by

J =∂(ηi)

∂(xj)=∂(∂ϑiS(x, ϑ))

∂(xj)= H(x, ϑ),

and therefore |det J | = |detH| ≥ 2 .

The integration of (4.3) gives const1

|det J | ·1

λnEn≤ const 1

· 1

λnEn.

The factor E−n goes away after (postponed) integration with respect toϑ in (4.2), since the size of ϑi-support of the integral kernel of τ is boundedby E ≡ minε, . Therefore, L1-norm of K(ττ∗)(x, y) with respect to x ory is bounded by const λ−n−1.

This results in the estimate ‖τ‖2 = ‖ττ∗‖ ≤ const λ−n−1, proving (3.10).

5. Proof of Lemma 3.1: almost orthogonality relations

Let us check the almost orthogonality relations for operators localized neardifferent lattice points in Rn× Rn. We write T for both T and T and donot distinguish them, since the argument stays the same. (We will not usethe key feature of T that on its support |detH| ≥ /2.)

Almost orthogonality relations:∥

∥T XΘ∗T

YW

∥

∥.

The integral kernel K(T XΘ∗T

YW)(ϑ,w) of T

XΘ∗T

YWis given by

(5.1)

∫

dnx ρ2(x/δ) %(ϑ/ε) %(w/ε) e−iλ(S(x,ϑ)−S(x,w))

×β(−1 detH(x, ϑ))β(−1 detH(x,w))

×χ(Σ−1x′ −X ′)χ(σ−1xn −Xn)χ(Σ−1x′ − Y ′)χ(σ−1xn − Yn)×χ(−1ϑ−Θ)χ(−1w −W ).

If ‖Y −X‖ ≥ 2√n, then (5.1) has empty x-support, and T

XΘ∗T

YW≡ 0.

Let us consider the case when |Wn −Θn| dominates. We will show thatfor sufficiently large values of |Wn −Θn|

|detH(x,w)− detH(x, ϑ)| ≥ 4,

then β(−1 detH(x, ϑ))β(−1 detH(x,w)) = 0, and also T XΘ

∗T YW

= 0.Let us put v = w − ϑ. For the difference |detH(x,w)− detH(x, ϑ)| we

have:

(5.2)

∫ 1

0

dtd

dtdetH(x, ϑ+tv) ≥

∫ 1

0

dt (|vn| |∂ϑn detH| − ‖v′‖ ‖∇ϑ′ detH‖) .

Note that the arc given by (x, ϑ+ t(w − ϑ)), 0 ≤ t ≤ 1, is inside the convexregion Dδε.

On the support of (5.1) the following inequalities are satisfied:

‖w′ − ϑ′‖ ≤ ‖W ′ −Θ′‖+ 2√n− 1, |wn − ϑn| ≥ |Wn −Θn| − 2.

Therefore, the expression under the integral sign in the right-hand side of(5.2) is greater than 4, if |Wn −Θn| is greater than some constant and if

(5.3) α |Wn −Θn| ≥ ‖W ′ −Θ′‖ supDδε

‖∇ϑ′ detH‖|∂ϑn detH| , α = 2−

14 .

This proves the almost orthogonality when |Wn −Θn| is relatively large.If (5.3) does not hold, then

(5.4) ‖W ′ −Θ′‖ supDδε

‖∇ϑ′ detH‖|∂ϑn detH| > α |Wn −Θn| ,

and due to the condition (2.13),

(2.13′)1

2≥ sup

Dδε

(

‖H′−1‖ ‖∇x′∂ϑnS‖)

supDδε

‖∇ϑ′ detH‖

infDδε

|∂ϑn detH| ,

we have:

1

2 ‖W ′ −Θ′‖ ≥ α |Wn −Θn| sup

Dδε

(

‖H′−1‖−1 ‖∇x′∂ϑnS‖)

.

For large enough ‖W −Θ‖, we can rewrite this inequality as

(5.5)1

2‖w′ − ϑ′‖ ≥ α2 |wn − ϑn| sup

Dδε

(

‖H′−1‖−1 ‖∇x′∂ϑnS‖)

.

We have spared one power of α for the errors in the relations

‖w′ − ϑ′‖ ≈ ‖W ′ −Θ′‖ , |wn − ϑn| ≈ |Wn −Θn| .

Now we employ the operator Lx, given by

Lx =1

iλ· (Sx(x,w)− Sx(x, ϑ)) · ∇x‖Sx(x,w)− Sx(x, ϑ)‖2

.

We will show that for relatively large values of ‖W ′ −Θ′‖ the denominatorof Lx is not less than const ‖w′ − ϑ′‖ (squared). It suffices to consider thefirst n− 1 components of the vector Sx(x, ϑ)− Sx(x,w).

(5.6)

‖∇x′S(x,w)−∇x′S(x, ϑ)‖ =∥

∥

∥

∥

∫ 1

0

dtd

dt∇x′S(x, ϑ+ v t)

∥

∥

∥

∥

=

∥

∥

∥

∥

∫ 1

0

∇x′∇ϑ′S(x, ϑ+ v t) · v′ +∇x′∂ϑnS(x, ϑ+ v t) vn dt

∥

∥

∥

∥

,

here v = w − ϑ. Let us mention that the integral in the right-hand side isentirely in Dδε.

We concentrate our attention on the first term in the integral in theright-hand side. The vector v′ does not depend on t, while the matrix∇x′∇ϑ′S(x, ϑ + v t) ≡ H

′t is a continuous function of t. The magnitude of

the vector H′tv′ is not less than ‖v′‖ inf

Dδε

min(H′). We can provide (possibly,

splitting the support of ψ(x, ϑ) into several smaller pieces) that everywherein Dδε the vector H

′tv′ remains in a small cone of magnitude φ, such that

cosφ ≥ α ≡ 2−1/4. Then the right-hand side of (5.6) is bounded from belowby

∫ 1

0

α ‖v′‖min(H′t)− |vn| ‖∇x′∂ϑnS(x, ϑ+ v t)‖ dt

≥∫ 1

0

α(1− α) ‖v′‖min(H′t) dt

+

∫ 1

0

(

α2 ‖v′‖min(H′t)− |vn| ‖∇x′∂ϑnS(x, ϑ+ v t)‖)

dt.

This gives the required bound from below for (5.6), since the first integral inthe right-hand side is greater than const ‖v′‖, and the second integral is non-negative due to (5.5). When using (5.5), we need to recall that min(H′(x, ϑ))

defined in (2.9) is equal to ‖H′(x, ϑ)−1‖−1.We insert L2n+1x into the integral kernel of T

XΘ∗T

YWand integrate by parts.

Each derivative ∇x gives at most max

δ−1,Σ−1

= ∆Σ

−1. This adds the

following factor to the integral kernel of T XΘ∗T

YW:

const1

|λ∆Σ ‖w′ − ϑ′‖|2n+1.

Note that when the derivative ∇x acts on the denominator of Lx, the ap-pearing factor is bounded by

const ‖w − ϑ‖‖Sx(x, ϑ)− Sx(x,w)‖

≤ const ‖W −Θ‖ ‖W ′ −Θ′‖ ≤ const ,

since the relation (5.4) implies that ‖W ′ −Θ′‖ ≥ const ‖W −Θ‖.We have for the L1-norm of (5.1) with respect to ϑ (or w):

(5.7)

∫

dϑ∣

∣K(T XΘ

∗T YW

)(ϑ,w)∣

∣ ≤∫

dx dϑ

suppT

XΘT

YW∗

const

|λ∆Σ ‖w′ − ϑ′‖|2n+1

≤ const∆Σ

n−1∆σ En (λ∆Σ ‖W −Θ‖)−n−1 .

We used the relation ‖w′ − ϑ‖′ ≈ ‖W ′ −Θ′‖ ≥ const ‖W −Θ‖, whichfollows from (5.4).

Since E ≤ , we may rewrite the factor before ‖W −Θ‖−2n−1 as

∆Σ

n−1∆σ En

(λ∆Σ)2n+1≤ const 1

λn∆σ

∆Σ

1

(λ∆Σ)n+1≤ const λ−n−1.

Here we used the inequality∆Σ ≥ const min

δ, Σ

≥ const δ ≥ const∆σ .Note that, according to (3.8) and (3.11), λ∆Σ ≥ λ∆σ2 ≥ λ∆σE ≥ 1.

Therefore, (5.7) is bounded by const λ−n−1 ‖W −Θ‖−2n−1. This provesthe estimate (3.12) for the operators T

XΘ∗T

YWand T

XΘ∗T

YWwhen ‖W ′ −Θ′‖

dominates.

Almost orthogonality relations:∥

∥T XΘT

YW

∗∥

∥. All the ideas are the same. We

consider the integral kernel K(T XΘT

YW

∗)(x, y) of the operatorT XΘT

YW

∗:

(5.8)

∫

dnϑ ρ(x/δ) ρ(y/δ) %2(ϑ/ε) eiλ(S(x,ϑ)−S(y,ϑ))

×β(−1 detH(x, ϑ))β(−1 detH(y, ϑ))

×χ(Σ−1x′ −X ′)χ(Σ−1y′ − Y ′)χ(σ−1xn −Xn)χ(σ−1yn − Yn)×χ(−1ϑ−Θ)χ(−1ϑ−W ).

If ‖W −Θ‖ > 2√n, then (5.8) has empty ϑ-support, and T

XΘTYW

∗ ≡ 0.If |Xn − Yn| is large, we want to find the condition for the difference

(5.9) |detH(y, ϑ)− detH(x, ϑ)|

to be greater than 4; then β(−1 detH(x, ϑ)) and β(−1 detH(y, ϑ)) haveno common support, and (5.8) is identically zero.

We rewrite (5.9) as

∫ 1

0

dtd

dtdetH(x+ tv, ϑ) ≥

∫ 1

0

dt (|vn| |∂xn detH| − ‖v′‖ ‖∇x′ detH‖) .

On the support of (5.8), we can write:

|vn| ≥ σ−1 (|Yn −Xn| − 2) , ‖v′‖ ≤ Σ−1

(

‖Y ′ −X ′‖+ 2√n− 1

)

.

Since we defined σ ≡ infDδε

|∂xn detH| , Σ ≡ supDδε

‖∇x detH‖ , the expression

(5.9) is greater than 4 if

(5.10) α |Xn − Yn| ≥ ‖X ′ − Y ′‖ .

This is all for the case when |Yn −Xn| is relatively large.If (5.10) does not hold, then we have:

(5.11) ‖X ′ − Y ′‖ ≥ α |Xn − Yn| ,

and hence

(5.12) ‖X − Y ‖ ≤ (1 + α−1) ‖X ′ − Y ′‖ .

We multiply (5.11) by (2.14),

(2.14′)1

2≥ sup

Dδε

(

‖H′−1‖ ‖∇ϑ′∂xnS‖)

supDδε

‖∇x′ detH‖

infDδε

|∂xn detH| ,

and obtain the relation

1

2Σ−1 ‖Y ′ −X ′‖ ≥ ασ−1 |Yn −Xn| sup

Dδε

(

‖H′−1‖ ‖∇ϑ′∂xnS‖)

.

We recall that on the support of (5.8)

‖v′‖ ≈ Σ−1 ‖Y ′ −X ′‖ , |vn| ≈ σ−1 |Yn −Xn| ,

and for sufficiently large ‖Y −X‖

(5.13)1

2‖v′‖ ≥ α2 |vn| sup

Dδε

(

‖H′−1‖ ‖∇ϑ′∂xnS‖)

.

We sacrificed one power of α to the errors in the approximations.We are going to employ the operator

Lϑ =1

iλ· (Sϑ(x, ϑ)− Sϑ(y, ϑ)) · ∇ϑ

‖Sϑ(x, ϑ)− Sϑ(y, ϑ)‖2,

which gives 1 when acting on the exponential in the integral kernel of (5.8).We claim that for sufficiently large values of ‖X ′ − Y ′‖ the denominator ofLϑ is bounded from below by const ‖x′ − y′‖ (squared). We can prove it usingthe relation (5.13) and repeating the argument after (5.6) (interchanging xand ϑ). We conclude:

‖Sϑ(x, ϑ)− Sϑ(y, ϑ)‖ ≥ (α− α2) ‖x′ − y′‖ infDδε

min(H′),

and this gives the desired bound from below for the denominator of Lϑ.Now we insert L2n+1

ϑinto the integral kernel of T

XΘTYW

∗ and integrate byparts. Each derivative ∇ϑ gives at most const max

ε−1, −1

≡ constE−1.When the derivative ∇ϑ falls on the denominator of Lϑ, the factor whichappears is also bounded by const −1:

const‖x− y‖‖x′ − y′‖ ≤ const

δ

Σ−1 ‖X ′ − Y ′‖ ≤ const δΣ−1.

Integration by parts in (5.8) yields the factor bounded by

const1

|λE ‖x′ − y′‖|2n+1.

We integrate the absolute value of (5.8) with respect to x (or y), obtainingthe L1-norm of K(T

XΘTYW

∗):

(5.14)

∫

dx∣

∣K(T XΘT

YW

∗)(x, y)∣

∣ ≤∫

dx dϑ

suppT

XΘT ∗

YW

const

|λE ‖x′ − y′‖|2n+1

≤ const∆Σ

n−1∆σ En(

λE Σ−1 ‖X − Y ‖

)−2n−1.

Note that ‖x′ − y′‖ ≈ Σ−1 ‖X ′ − Y ′‖ ≥ constΣ−1 ‖X − Y ‖ , due to the

inequality (5.12).Since ∆Σ ≥ ∆σ and λE Σ

−1 ≥ λE ∆Σ ≥ λE∆σ ≥ 1, we see that

∆Σ

n−1∆σ En

(λEΣ−1)2n+1≤ const ∆Σ

n−1∆σ En

(λE∆Σ)n ≤ const λ−n−1.

This results in the bound const λ−n−1 ‖X − Y ‖−2n−1 for (5.14), provingthe estimate (3.12) for the operators T

XΘTYW

∗ and T XΘT

YW

∗ when ‖X ′ − Y ′‖dominates.

This concludes the proof of Lemma 3.1 and of Theorem 2.2. ¤

6. Applications to scattering theory

Now we return to the oscillatory integral operator (1.6) with the phasefunction (1.5):

(6.1) Tλu(x) =

∫

Sndvolϑ e

iλ〈r(x),ω(ϑ)〉 ψ(x, ϑ)u(ϑ).

The variables x and ϑ are some local coordinates on B and Sn. The vectorr ∈ Rn+1 has its end in the point x on the hypersurface B, and ω is a vectorpointing to ϑ on Sn.

Local coordinates on B and on Sn. Let us localize the integral operator(6.1) near some critical point (xo, ϑo). We use the orthogonal projectionpL

: B → TroB to parameterize B. We take a Euclidean coordinate sys-

tem (x1, . . . , xn, xn+1) centered at ro, so that the direction of the axis xn+1coincides with the direction of the inner normal to B at ro. The axes xi,i = 1, . . . , n are thus in the tangent plane Tro

B. The hypersurface B islocally parameterized by x = (x1, . . . , xn) ∈ Tro

B ≡ RnL:

xn+1 = b (x) ≥ 0, x = (x1, . . . , xn) ∈ U ⊂ RnL.

In the same way, we use the orthogonal projection pR

: Sn → TϑoSn ≡

RnR

to parameterize the sphere. We take a Euclidean coordinate system(ϑ1, . . . , ϑn, ϑn+1) centered at ωo, with the axis ϑn+1 being the inner normalto Sn at the point ωo. The sphere is then given locally by

ϑn+1 = g(ϑ) ≡ 1−√

1− ‖ϑ‖2 ' ‖ϑ‖2

2, here ‖ϑ‖2 = ϑ21 + · · ·+ ϑ2n.

Since (ro, ωo) is a critical point, the planes TroB and TωoSn are mutually

orthogonal. We choose the directions of the axes xi and ϑi so that thedirection of ϑn is the same as that of xn+1, xn is opposite to ϑn+1, and thedirections xi′ and ϑi′ coincide (i′ = 1, . . . , n− 1).

The dot product 〈r, ω〉 is then given by x′ · ϑ′ + ϑnxn+1 − xnϑn+1, andtherefore

(6.2) S(x, ϑ) = 〈r, ω〉 = x′ · ϑ′ + ϑnb (x)− xng(ϑ).

The determinant of the mixed Hessian of this phase function is given by

(6.3) detH(x, ϑ) = bxn(x)− gϑn(ϑ) + bx′(x) · gϑ′(ϑ).

The non-singular part of the mixed Hessian is a unit matrix:

H′ = ∇x′∇ϑ′S = IRn−1 ,

hence in (2.9) min(H′) = 1.

Convexity of the map πR. Let us show that the map π

R|ϑcorresponding to

the phase function (6.2) is pseudoconvex.This map is defined as

πR|ϑ: x 7→ Sϑ(x, ϑ) = ∇ϑ〈r(x), ω(ϑ)〉.

If we use the standard scalar product 〈 , 〉 in Rn+1 to identify the tangentand cotangent fibers, Tω(ϑ)Sn ∼= T ∗ω(ϑ)S

n, then πRis given geometrically by

the orthogonal projection Πω(ϑ) from B onto Tω(ϑ)Sn:

BΠω(ϑ)−→ Tω(ϑ)Sn, r(x) 7→ r(x)− ω(ϑ)〈r(x), ω(ϑ)〉.

If the hypersurface B is convex (not necessarily strictly convex), then oneach connected set O ⊂ B, for any r1, r2 ∈ O,

(6.4) ‖Πω(r2 − r1)‖TωSn ≥ ‖r2 − r1‖Rn+1 infr∈O|〈nr, ω〉| .

This is the key inequality. Since 〈nr, ω〉 is approximately the determinant ofthe Jacobian of Πω, (6.4) suggests that the map Πω is pseudoconvex.

Using the orthogonal projections pLand p

R,

p−1L

: RnL≡ Tro

B → B and pR: Sn → TωoSn ≡ Rn

R,

we write the map πR|ϑ: Rn

L→ T ∗

ϑRn

R∼= TϑRn

Ras

πR|ϑ: x

p−1L7−→ r(x)

Πω(ϑ)7−→ r(x)− ω(ϑ)〈r(x), ω(ϑ)〉 dpR7−→ Sϑ(x, ϑ) ∈ TϑRnR.

The maps p−1L

and dpR

are diffeomorphisms, and we expect that πR|ϑ

ispseudoconvex as a composition of pseudoconvex maps.

Let us write this in more detail.We substitute into the relation (6.4) r1 = r(x), r2 = r(y), ω = ω(ϑ):

∥

∥Πω(ϑ)(r(y)− r(x))∥

∥

Tω(ϑ)Sn≥ ‖r(y)− r(x)‖Rn+1 inf

z∈O

∣

∣〈nr(z), ω〉∣

∣ .

Here O = pL(O) ⊂ Rn

L.

Then we take into account the relations

‖dpR(∗)‖Rn

R≥ |〈ω, ωo〉| ‖∗‖TωSn and ‖r(y)− r(x)‖Rn+1 ≥ ‖y − x‖Rn

L

and obtain:

(6.5) ‖Sϑ(y, ϑ)− Sϑ(x, ϑ)‖T∗

ϑRnR≥ ‖y − x‖Rn

L|〈ω, ωo〉| inf

z∈O

∣

∣〈nr(z), ω〉∣

∣ .

Considering the projections RnL→ Tr(x)B → Tω(ϑ)Sn → Rn

R, we conclude

that

(6.6) detH(x, ϑ) =〈ω(ϑ), ωo〉〈nr(x),nro

〉 〈nr(x), ω(ϑ)〉.

We have localized (6.1) near the point (xo, ϑo) = (0, 0), and therefore

〈ω(ϑ), ωo〉 = 〈ω(ϑ), ω(ϑo)〉 ' 1, 〈nr(x),nro〉 ' 1.

We rewrite (6.5) as

(6.5′) ‖Sϑ(x, ϑ)− Sϑ(y, ϑ)‖T∗

ϑRnR≥ 1

2‖x− y‖Rn

Linfz∈O|detH(z, ϑ)| .

Note that since |detH′| ≥ const , min(H) ∼ |detH| .

This proves pseudoconvexity of the map πR|ϑ. ¤

Conditions (2.12)-(2.14) of Theorem 2.2. We want to rewrite the technicalassumptions of Theorem 2.2 for the phase function (6.2).

Let us consider a hypersurface B, locally given by xn+1 = b (x), x =(x1, . . . , xn), b ∈ C2(Rn

L).

The condition (2.13) is given by

(6.7) sup |∇x′b|sup ‖∇ϑ′(−∂ϑng +∇x′b · ∇ϑ′g)‖inf |∂ϑn(−∂ϑng +∇x′b · ∇ϑ′g)|

≤ 1

2.

Note that in the denominator ∂2ϑng ≈ 1.

Under a very weak assumption

(6.8) ‖∇xb (x)‖ ≤ ‖x‖ ,

the inequality (6.7) is satisfied if ‖x‖ and ‖ϑ‖ are reasonably small:

(6.9) ‖x‖ ≤ 1/6, ‖ϑ‖ ≤ 1/6.

The main restriction on ‖ϑ‖ arises from the condition (2.14). This condi-tion amounts to

(6.10) sup ‖∇ϑ′g‖sup ‖∇x′(∂xnb+∇x′b · ∇ϑ′g)‖inf |∂xn(∂xnb+∇x′b · ∇ϑ′g)|

≤ 1

2.

Calculation gives the following sufficient restriction:

(6.11) ‖ϑ‖ ≤ 1

6

inf ∂2xnb

sup ‖∇x∇xb‖.

Here ‖∇x∇xb‖ is the norm of the Hessian of b, which is considered as aquadratic form on Rn.

At last, as long as x and ϑ are in the limits specified by (6.9) and (6.11),we have:

(6.12) |∂ϑn detH| > 1/2, |∂xn detH| > ∂2xnb(x)/2.

We have reformulated all the conditions of Theorem 2.2, to adapt themto the oscillatory integral operator (6.1). Now we are ready to proceed toderiving the estimates on its norm.

Hypersurface with asymptotically small principal curvatures. We start withthe case when the principal curvatures of the hypersurface B are asymptoti-cally small.

Let B be given locally by the graph xn+1 = b (x1, . . . , xn) ≥ 0, b(0) = 0,and let the Hessian [∂xi∂xj b] be isotropic:

(6.13) uiuj∂2b (x)

∂xi∂xj≥ σ,

∣

∣

∣

∣

uivj∂2b (x)

∂xi∂xj

∣

∣

∣

∣

≤ Cσ, x ∈ U ⊂ Rn,

for any unit vectors u, v in Rn. C > 1 is some constant. These conditionssay that all sectional curvatures are of magnitude σ.

We would like to know the asymptotic behavior of the operator (6.1) whenσ is asymptotically small. Of course, σ in (6.13) is going to represent theprincipal curvature of the fold π

Rin (2.12).

Without loss of generality, we assume that Cσ < 1.

Proposition 6.1 (Asymptotically small sectional curvature).Consider an integral operator Tλ : E ′(Sn)→ D′(B),

Tλu(x) =

∫

Sndvolϑ e

iλ〈r(x),ω(ϑ)〉 ψ(x, ϑ)u(ϑ), ψ ∈ C2n+1(B× Sn),

with B a smooth strictly convex hypersurface in Rn+1 (locally a graph ofC2n+2-function).

Let all sectional curvatures of B be of the same magnitude σ > 0, σ ¿ 1.Then Tλ is continuous from L2(Sn) to L2loc(B) with the norm

(6.14) ‖Tλ‖ ≤ const λ−n2+

16σ−

16 .

Corollary 6.2. Let K be a strictly convex compact domain in Rn+1 witha smooth boundary B. Then the Radon Transform R in (1.1), localized inthe region where the sectional curvatures of B are of magnitude σ, acts con-tinuously from Hs(R) ⊗ L2(Sn) to Hs+n

2−16 (R) ⊗ L2loc(B), with the norm

‖R‖ ≤ const σ− 16 .R also acts continuously from Hs(R × Sn) to Hs+n

2−16 (R × B), with the

norm const(s)σ−16 .

The assumptions of Proposition 6.1 imply that locally the hypersurface Bis a graph of a smooth function b, which satisfies (6.13). As a matter of fact,we do not need to require that all sectional curvatures of B be of the samemagnitude, and the second condition in (6.13) is not necessary.

Example: ball of radius R = σ−1. Let us first give an example of such asituation. We take B = SnR and consider the estimate on the oscillatoryintegral operator

Tλu(r) =

∫

Sn1

dvolϑ eiλ〈r,ω〉u(ϑ), r ∈ SnR.

The operator Tλ, considered as an operator to the space of functions de-fined on a unit sphere, is written as

Tλu(nr) =

∫

Sn1

dvolϑ eiλR〈nr,ω〉u(ϑ), nr =

r

R∈ Sn1 ,

and the estimate (1.7) yields ‖Tλ‖L2(Sn1 )→L2(Sn1 )≤ const (λR)−n

2+16 .

Using this estimate, together with

∫

SnR

dvol = Rn

∫

Sn1

dvol, we arrive at

∫

SnR

dvol |Tλu(r)|2 = Rn

∫

Sn1

dvol |Tλu(nr)|2 ≤ Rnconst (λR)−n+13 ‖u‖2L2(Sn) .

Thus,

‖Tλ‖L2(Sn1 )→L2(SnR) ≤ const λ−

n2+

16R

16 ,

in an agreement with (6.14).

The increase in the norm could be connected with both the “more criticalbehavior” at critical points (as the curvature gets smaller) and the increaseof the area of B = SnR. As a matter of fact, the increase of ‖Tλ‖ as R → ∞is a sheer consequence of the diminishing of a curvature.

Proof of Proposition 6.1. We are going to apply Theorem 2.2 to the operatorTλ in (6.1), localized near a critical point (xo, ϑo), 〈nr(xo), ω(ϑo)〉 = 0.

Since we assume that b(x) in (6.2) satisfies the condition (6.13), the rela-tions (6.12) yield

(6.15) |∂ϑn detH| > 1/2, |∂xn detH| > σ/2.

The restrictions (6.9) and (6.11) on ‖x‖ and ‖ϑ‖ are

(6.16) ‖x‖ ≤ 1/6, ‖ϑ‖ ≤ 1/ 6C,

with the constant C from (6.13). Formally, we take δ = 1/6, ε = 1/ 6C.According to Theorem 2.2, we have:

‖Tλ‖ ≤ const λ−n2+

16σ−

16 . ¤

Hypersurface with curvature vanishing at a point. Now let us consider thecase when the hypersurface B admits a tangent plane with a contact of preciseorder k > 1. Then B is given locally by

xn+1 = b (x1, . . . , xn) ∼ ‖x‖k+1 , ‖x‖2 = x21 + · · ·+ x2n, x ∈ U ⊂ Rn.

The principal curvatures behave like σ ∼ ‖x‖k−1 as ‖x‖ approaches 0. The

Gaussian curvature of the hypersurface B vanishes ∼ ‖x‖n(k−1).The definition of a contact of precise order k is given by (1.3): for any

unit vectors u, v ∈ Rn, the Hessian of the function b (x) satisfies

(1.3′) uiuj∂2b (x)

∂xi∂xj≥ c ‖x‖k−1 ,

∣

∣

∣

∣

uivj∂2b (x)

∂xi∂xj

∣

∣

∣

∣

≤ C ‖x‖k−1 .

Proposition 6.3 (Curvature uniformly vanishing at a point).Consider an integral operator Tλ : E ′(Sn)→ D′(B),

Tλu(x) =

∫

Sndvolϑ e

iλ〈r,ω〉 ψ(x, ϑ)u(ϑ), ψ ∈ C2n+1(B × Sn),

with B a smooth convex hypersurface in Rn+1 (locally a graph of C2n+2-function).

Let the Gaussian curvature of B vanish at an isolated point ro on thesupport of ψ. If the tangent plane at ro has a contact of precise order k > 1with B, then Tλ acts continuously from L2(Sn) to L2loc(B), with the norm

(6.17) ‖Tλ‖ ≤ const λ−n2+

12

k2k+1 .

Proof of Proposition 6.3. We use the same local coordinates x as in (1.3), sothat the origin x = 0 in Rn

Lcorresponds to the critical point ro ∈ B.

We will need a fine spatial partition of unity, as x approaches the origin.Let us introduce a function ρ(x) ∈ C∞o (Rn), supported for 2 ≤ ‖x‖ ≤ 8, suchthat

∑

j≥0

ρ(2jx) ≡ 1 for ‖x‖ ≤ 2.

We split ρ(x) into a sum of several functions ρ(x) =∑

ρa(x), a = 1 . . . A,supported in unit balls centered at some points pa ∈ supp ρ, 2 ≤ ‖pa‖ ≤ 8:ρa ∈ C∞o (Bn

1 (pa)). In each ball Bn1 (pa), we have:

‖x‖ ≥ 1, sup ‖x‖ / inf ‖x‖ ≤ 3.

Decompose Tλ as follows:

(6.19) Tλ =∑

δ≤1

A∑

a=1

ρa(x/δ) Tλ, δ = 2−j , j ≥ 0.

We will apply Theorem 2.2 for the operators ρa(x/δ) Tλ.The condition (1.3) gives ∂2xnb ≥ c ‖x‖

k−1, ‖∇x∇xb‖ ≤ C ‖x‖k−1, and the

restriction (6.11) on ‖ϑ‖ becomes

‖ϑ‖ ≤ ε ≡ c

6C· 1

3k−1.

Note that the value of ε is only finitely small, so that we could cover thesphere by finitely many neighborhoods of this size.

For each individual operator ρa(x/δ) Tλ, we need to consider the setDδε(pa/δ, 0) ≡ Bn

δ (pa/δ)× Bnε (0), with ε as above. On this set, ‖x‖ ≥ δ.

The principal curvature of the fold πRon Dδε(pa/δ, 0) is given by (6.12):

(6.20) σ(δ) >1

2c δk−1.

Theorem 2.2 gives the following bounds on ‖ρa(x/δ) Tλ‖:

(6.21) const λ−n2+

14 δ

14 and const λ−

n2+

16 σ(δ)−

16 .

Now we can evaluate the operator norm of (6.19). Let us find the value ofδ when the two estimates (6.21) on ‖ρa(x/δ) Tλ‖ clutch together:

λ14 δ

14 = λ

16 δ−

k−16 .

This gives δ = λ−1

2k+1 , and we conclude that ‖Tλ‖ ≤ const λ−n2+

12

k2k+1 . ¤

Theorem 1.1 formulated in the Introduction is a consequence of Proposi-tion 6.3 and of (1.7).

ACKNOWLEDGMENTS

The author is indebted to his advisor, Professor D.H. Phong, for formu-lating the original problem and for help during the work. Also, the author isgrateful to A. Greenleaf, R.B. Melrose, A. Seeger, C. Sogge, and E.M. Steinfor their attention and interest in the research.

Author specially thanks S. Cuccagna for very helpful discussions and forcorrections of an earlier draft.

References

[1971] L. Hormander, Fourier integral operators, Acta Math. 127 (1971), 79–183.[1985] , The analysis of linear partial differential operators III, IV, Springer-

Verlag, 1985.[1985] R.B. Melrose and M.E. Taylor, Near peak scattering and the corrected Kirchhoff

approximation for a convex obstacle, Adv. in Math. 55 (1985), 242–315.

[1990] Y.B. Pan and C.D. Sogge, Oscillatory integrals associated to folding canonical

relations, Colloq. Math. 61 (1990), 413–419.[1991] Y.B. Pan, Uniform estimates for oscillatory integral operators, J. Funct. Anal.

100 (1991), 207–220.[1991] D.H. Phong and E.M. Stein, Radon transform and torsion, Internat. Math. Res.

Notices 4 (1991), 49–60.

[1993] E.M. Stein, Harmonic Analysis: real-variable methods, orthogonality, and oscil-

latory integrals, Princeton Univ. Press, 1993.[1994] A. Greenleaf and A. Seeger, Fourier integral operators with fold singularities, J.

Reine Angew. Math. 455 (1994), 35–56.[1994] D.H. Phong, Singular integrals and Fourier integral operators, Essays on Fourier

Analysis in honor of Elias M. Stein (C. Fefferman, R. Fefferman and S. Wainger,

eds.), Princeton Univ. Press, 1994, pp. 287–320.[1994] D.H. Phong and E.M. Stein, Models of degenerate Fourier integral operators and

Radon transforms, Ann. of Math. 140 (1994), 703–722.

[1995] S. Cuccagna, L2 estimates for averaging operators along curves with two sided k

fold singularities, Preprint (1995).