1

Basic statistical concepts and least-squares. Sat05_61.ppt, 2005-11-28.

1. Statistical concepts2. Distributions• Normal-distribution

2. Linearizing3. Least-squares• The overdetermined problem• The underdetermined problem

2

Histogram.Describes distribution of repeated observationsAt different times or places !

Distribution of global 10 mean gravity anomalies

3

Statistic:

Distributions describes not only random events !

We use statistiscal description for ”deterministic” quantities as well as on random quantities.

Deterministic quantities may look as if they have a normal distribution !

4

”Event”

Basic concept:Measured distance, temperature, gravity, …

Mapping:

Stocastisc variabel .In mathematics: functional, H – function-space – maybe

Hilbertspace.Gravity acceleration in point P: mapping of the space of all

possible gravity-potentials to the real axis.

RHX :

5

Probability-density , f(x):

What is the probability P that the value is in a specific interval:

dx f(x) = b) x P(ab

a

6

Mean and variance, Estimation-operator E:

))E(( :momentth n'

))(()()(

Variance

)()(x

:value-Mean

-

222x

-

nxx

xxEdxxfxx

xEdxxfx

7

Variance-covariance in space of several dimensions:

Mean value and variances:

jijjiiij

ijxx

dxdxxxxx ))((

2

8

Correlation and covariance-propagation:Correlation between two quantities: = 0: independent.

Due to linearity

1,1jjii

ijij

)()()( ybExaEYbXaE

9

Mean-value and variance of vector

If X and A0 are vectors of dimension n and A is an n x m matrix, then

The inverse P = generally denoted the weight-matrix

TX

TY AAYEYYEyE

XEAAYEXAAY

)))(())(((

)()( 00

10

Distribution of the sum of 2 numbers:

• Exampel Here n = 2 and m = 1. We regard the sum of 2 observations:

• What is the variance, if we regard the difference between two observations ?

2211YY21

22

11X0

+ = ,X + X = Y0

0 = 1}, ,{1 = A {0}, = A

11

Normal-distribution

1-dimensional quantity has a normal distribution if

• Vektor of simultaneously normal-distributed quantities if

• n-dimensional normal distribution.

e2

1 = f(x) 2)/()E(X)--(x

xx

xx2

e )()(2

1 = )x,...,xF( E(X))/2-(X P E(X))--(X

X1/2n/2n1

T

det

12

Covarians-propagation in several dimensions:

X: n-dimensional, normally distributed,D nxm matrix , then

Z=DZ also normal distributed, E(Z)=D E(X)

TXZ DDZE )( 2

13

Estimate of mean, variance etc

products ofnumber

/)ˆ)(ˆ(),cov( :Covariance

1)ˆ( :deviation-Standard

/ˆ :Mean

1

1

2

1

n

nyyxxyx

nxx

nxx

n

iii

n

i

ix

n

ni

14

Covarians function

If the covariance COV(x,y) is a function of x,y then we have a

Covarians-functionMay be a function of• Time-difference (stationary)• Spherical Distance, ψ on the unit-sphere (isotrope)

15

Normally distributed data and resultat.

If data are normaly dsitributed, then the resultats are also normaly distributed

If they are linearily related !

We must linearize – TAYLOR-Development with only 0 and 1. order terms.

Advantage: we may interprete error-distributions.

16

Distributions in infinite-dimensional spaces

V( P) element in separable Hilbert-space:

Normal distributed with sum of variances finite ! ijij

ij

ijiji

i

ij

GMCVX

V

PVCGMPV

)( :(example) variableStochastic

functions-base orthogonal

),()(0

17

Stochastic process.

What is the probability P for the event is located in a specific interval

Exampel: What is the probability that gravity in Buddinge lies in between -20 and 20 mgal and that gravity in Rockefeller lies in the same interval

),( 21 dXcbXaP

18

Stokastisc process in HilbertspaceWhat is the mean value and variance of ”the Evaluation-

functional”,

20

2

20

0

22

)(:

)()()(

0)()()(

0)(,)(,0)(

),()(

iiP

iiiiqp

iiiP

jiiii

P

EvEVariance

QVPVEvEvE

PVXEEvE

XXEXEXEwith

PTTEv

19

Covariance function of stationary time-series.

Covariance-function depends only on |x-y|

Variances called ”Power-spectrum”.

y))-(i(x

= f(y)) E(f(x) = y)COV(x,

2i

N

=0i

2i

N

=0i= y)) (ix) (i + y) (ix) (i(

cos

sinsincoscos

2

i2

i2

iii

N

=0i = ))bE(( = ))aE(( x)),

Ni( b + x)

Ni( a( = f(x) 2sin2cos

20

Covariance function – gravity-potential.

• Suppose Xij normal-distributed with the same variance for constant ”i”.

1)+/(2i= = ))CRGM(E( 2

i2ij

2ij

ts.coefficien normalizedfully

radius,mean sEarth' R longitude, latitude,

),,(),,()(2

ij

i

i

ijij

i

ij

C

VrRC

rGMrTPT

21

Isotropic Covariance-function for Gravity potential

distance. spherical

spolynomial Legendre ),(cos'

)','(),('

)12/(

))(),((),(

2

122

2

122

iii

i

i

iijij

ii

iji

PPrrR

VVrrRi

QTPTEQPCOV

22

Linearizering: why ?

We want to find best estimate (X) for m quantities from n observations (L).

Data normal-distributed, implies result normaly distributed, if there is a linear relationship.

If m > n there exist an optimal metode for estimating X:

Metode of Least-Squares

23

Linearizing – Taylor-development.

If non-linear:

Start-værdi (skøn) for X kaldes X1

Taylor-development with 0 og 1. order terms after changing the order

1010 ,),( XXxLLyXL

|}X

{ = A x, A = v +y X 1

.parameters ofFunction noisensobservatioor )(

XL

24

Covariance-matrix for linearizered quantities

If measurements independently normal distributed with varians-covariance

Then the resultatet y normal-dsitributed with variance-covarians:

ij

Tijy AA

25

Linearizing the distance-equation.

Linearized based on coordinates

)X ,X(

X-X = |X

orden.-2 af led + )X-X(|X

+ )X ,X( = )X (X,

01

0i1iX

1i

1iiX1i

3

=1i010

1

1

2033

2022

20110 )()()(),( XXXXXXXX

),,( 131211 XXX

26

On Matrix form:

If

3 equations with 3 un-knowns !

)( 0iii XXdX

yxAorcomputedobserved

XXXXdXX

XXi

T

Xi

,

),(),(),(010

0

0

27

Numerical-example

If (X11, X12,X13) = ( 3496719 m, 743242 m, 5264456 m).Satellite: (19882818.3, -4007732.6 , 17137390.1) Computed distance: 20785633.8 mMeasured distance: 20785631.1 m

((3496719.0-19882818.3)dX1 + (743242.0-4007732.6) dX2+(5264456 .0-17137390.1) dX3)/20785633.8 =

( 20785631.1 - 20785633.8) or:

-0.7883 dX1 -0.1571 dX2 + 1.7083 dX3 = -2.7

28

Linearizing in Physical Geodesy based on T=W-U

In function-spaces the Normal-potential may be regarded as a 0-order term in a Taylor-development.We may differentiate in Metric space (Frechet-derivative).

anomaly)(gravity 2anomaly-height /

Trdr

dTg

T

29

Method of Least-Square. Over-determined problem.

More observations than parameters or quantities which must be estimated:

Examples: GPS-observations, where we stay at the same

place (static)We want coordinates of one or more points.

Now we suppose that the unknowns are m linearily independent quantities !

30

Least-squares = Adjustment.

• Observation-equations:• We want a solution so that

Differentiation:

•

xAy

minimum(x) = )x A-(y x) A -(y = v v T-1y

T-1y

ningerne)(Normalligy A )A A( = x

y A = x A A

0 = x)A()A(2 - )(A y2

0 = )Ax -(y Ax) -(y xd

d

1-y

T1-1-y

T

1-y

1-y

T

ii1-

yT

ii1-

yi

T1-y

i

31

Metod of Last-Squares. Variance-covariance.

)A A(

= )A)A A( A )A A(

=

1-1-y

T

T1-y

T1-1-y

Ty

1-y

T1-1-y

T

x

32

Metod of Least-Squares. Linear problem.

Gravity observations:

H, g=981600.15 +/-0.02 mgal

G

I10.52+/-0.03

12.11+/-0.03

-22.7+/-0.03

33

Observations-equations..

22.70-

10.52

12.11

981600.15

=

g

g

g

1- 0 1

1 1- 0

0 1 1-

0 0 1

I

H

G

1- 0 1

1 1- 0

0 1 1-

0 0 1

030. 0.00 0.00 0.00

0.00 030. 0.00 0.00

0.00 0.00 030. 0.00

0.00 0.00 0.00 020.

1- 1 0 0

0 1- 1 0

1 0 1- 1

=

2

2

2

2 -1

1-x

34

Method of Least-Squares. Over-determined problem.

Compute the varianc-covariance-matrix

35

Method of Least-Squares.

Optimal if observations are normaly distributed + Linear relationship !

Works anyway if they are not normally distributed !And the linear relationship may be improved using

iteration.

Last resultat used as a new Taylor-point.

Exampel: A GPS receiver at start.

36

Metode of Least-Squares. Under-determined problem.

We have fewer observations than parameters: gravity-field, magnetic field, global temperature or pressure distribution.

We chose a finite dimensional sub-space, dimension equal to or smaller than number of observations.

Two possibilities (may be combined):• We want ”smoothest solution” = minimum norm• We want solution, which agree as best as possible with

data, considering the noise in the data

37

Method of Least-Squares. Under-determined problem.

Initially we look for finite-dimensional space so the solution in a variable point Pi becomes a linear-combination of the observations yj:

If stocastisk process, we want the ”interpolation-error” minimalized

y = x jij

n

j=1i ~

)minimum( = )yyE( y 2- )xE( =

)y - E(x = )x - E(x

ijiji

n

j=1

n

=1iii

n

=1i

2

2ii

n

=1i

2

+ )E(x

~

38

Method of Least-Squares. Under-determined problem.

Covariances:Using differentiation:

Error-variance:

)xE( = C ),yE(x = C ),yyE( = C 20iPijiij

y)C()C( = x -1ijPi ~

C = E(xy) ,C = ))y()yE(( 0, = E(y) 0, = E(x) med

C C C -

= C C )yE( C C +

y)E(x CC 2 - )xE( = ))x - E((x

PjijT

ji

Pjij1-

PiT2

x

PjT

ij1-2

ij1-

Pi

ij-1

PiT22

~

39

Method of Least-Squares. Gravity-prediction.

Example:

Covarianses: COV(0 km)= 100 mgal2

COV(10 km)= 60 mgal2

COV(8 km)= 80 mgal2

COV(4 km)= 90 mgal2

P6 mgal

Q10 mgal

R

10 km

8 km 4 km

40

Method of Least-Squares. Gravity prediction.

Continued:

Compute the error-estimate for the anomaly in R.

mgal 9 = 6

10

100 60

60 100 80 90 = g

-1

R

~

41

Least-Squares Collocation.

Also called: optimal linear estimation

For gravity field: name has origin from solution of differential-equations, where initial values are maintained.

Functional-analytic version by Krarup (1969)

Kriging, where variogram is used closely connected to collocation.

42

Least-Squares collocation.

We need covariances – but we only have one Earth.

Rotate Earth around gravity centre and we get (conceptually) a new Earth.

Covariance-function supposed only to be dependent on spherical distance and distance from centre.

For each distance-interval one finds pair of points, of which the product of the associated observations is formed and accumulated. The covariance is the mean value of the product-sum.

43

Covarians-function for gravity anomalies, r=R.

distance. spherical fixed

,),','(),,()(

),(2

2

Earthazimuth

i

i

ijij

i

ij

ddRgRgC

VrRC

rGMg

44

Covarians function for gravity-anomalies:

Different models for degree-variances (Power-spectrum):

Kaula, 1959, (but gravity get infinite variance)Tscherning & Rapp, 1974 (variance finite).

ser)gradvarian mali(tyngdeano ,C )1-(iR

GM =

),(P = )C(

ij2

j

-ij=2

22i

i2i

2=i

cos