G-inverse and Solution of System of Equations and their Applications in Statistics

PROFESSOR DR. S. K. BHATTACHARJEEDEPARTMENT OF STATISTICS

UNIVERSITY OF RAJSHAHI, BANGLADESH

G-inverse and Solution of System of Equations

and their Applications in Statistics

System of Linear Equations

Linear Combination of Vectors

One extremely helpful view is that each unknown is a weight for a column vector in a linear combination.

Vector Form

The vector equation is equivalent to a matrix equation of the form

where A is an m×n matrix, x is a column vector with n entries, and b is a column vector with m entries.

Solution of Linear System

A solution of a linear system is an assignment of values to the variables x1, x2, ..., xn such that each of the equations is satisfied. The set of all possible solutions is called the solution set.

A linear system may behave in any one of three possible ways:

The system has infinitely many solutions.The system has a single unique solution.The system has no solution.

Geometric InterpretationFor a system involving two variables (x and y),

each linear equation determines a line on the xy-plane. Because a solution to a linear system must satisfy all of the equations, the solution set is the intersection of these lines, and is hence either a line, a single point, or the empty set.

For three variables, each linear equation determines a plane in three-dimensional space, and the solution set is the intersection of these planes. Thus the solution set may be a plane, a line, a single point, or the empty set.

For n variables, each linear equations determines a hyperplane in n-dimensional space. The solution set is the intersection of these hyperplanes, which may be a flat of any dimension

Solution of Two Vectors

Solution of Three EquationsBelow is a picture of three planes that have no solution. There is no single point at which all three planes intersect, therefore this system has no solution.

Solution of Three Equations

Each plane intersects the other two planes. However, there is no single point at which all three planes meet. Therefore, the system of 3 variable equations below has no solution.

Solution of Three Equations One Solution of three variable systems

I f the three planes intersect as pictured below then the three variable system has 1 point in common, and a single solution represented by the black point below.

Solution of Three Equations I nfinite Solutions of three variable systems

I f the three planes intersect as pictured below then the three variable system has a line of intersection and therefore an infinite number of solutions.

Solution of Two Equations in Three Variables

The solution set for two equations in three variables is usually a line.

Solution of Linear System In general, the behavior of a linear system is

determined by the relationship between the number of equations and the number of unknowns:

Usually, a system with fewer equations than unknowns has infinitely many solutions. Such a system is also known as an underdetermined system.

Usually, a system with the same number of equations and unknowns has a single unique solution.

Usually, a system with more equations than unknowns has no solution. Such a system is also known as an overdetermined system.

Examples The system has exactly 1 solution.

Systems have 1 and only 1 solution when the two lines have different slope. Think about it, if the two lines have different slopes then eventually at some point they must meet. After all the lines are not parallel.

system has no solutions Systems have no solution when the lines are parallel (ie have the same

slope) and the lines have different y-intercepts. As an example look at the following two lines

Line 1: y = 5x +13 Line 2: y = 5x + 12

The system has infinite solutions Systems have infinite solutions when the lines are parallel and the lines

have the same y-intercept. If two lines have the same slope (ie are parallel) and the same y-intercept, they are actually the same exact line. In other words, systems have infinite solutions when the two lines are the same line! As an example consider the following two lines

Line 1: y = x +3 Line 2: 2y = 2x +6

These two lines are exactly the same line. If you multiply line 1 by two you get line 2.

Solution of Linear System

The following pictures illustrate this in the case of two variables:

One Equation Two Equations Three Equations

Two Variables Three Eq.

The equations x − 2y = −1, 3x + 5y = 8, and 4x + 3y = 7 are not linearly independent.

Example

The equations 3x + 2y = 6 and 3x + 2y = 12 are inconsistent.

Methods of SolutionThe Methods of finding the solution to

systems of linear equations: graph : by looking at where lines intersect (meet)

on a graph algebraic equation : by setting the equations of

the system equal to each other then solving this equation.

substitution : by solving for one of the variables and substituting its value in to the other equation.

Elimination : Elimination involves algebraic manipulations of two or more equations. The end goal is to eliminate a variable by creating opposite coefficients (The examples below should clarify this straightforward approach).

Graphical Method

The Graph Method On the left, the system of linear equations is the following two lines:

y=x+1 y=2x

What is the solution? answer: The point (1,2) is where the two lines intersect.

Algebraic Equation Method

The Algebraic Equation Method

Let's take another look at the system of equations from above:

y=2x+1 y=4x-1

By examining the graph we can see that the point of intersection, or the solution, is the point (1,3) where the lines intersected.

Steps for the algebraic method: make sure that each linear equation is

reduced to slope intercept form o (ie y=3x+2 is good but 2y=6x+4 is NOT)

set the two equations equal to each other

o 2x+1=4x-1 Solve for X

o 2x+1=4x-1 o 2=2x o x= 1

insert x value into either equation to determine y coordinate of solution

o 4(1)-1=3 The solution is the ordered pair you've

just calculated o (1,3)

Substitution MethodThe Substitution Method  The substitution method involves algebraic

substitution of one equation into a variable of the other.

A quick refresher on algebraic substitution:

Refresher:Substitution  Equation 1 : x = 5 Equation 2: y = x +2 How to Substitute

1) Use equation 1( x= 5) to substitute 5 for x in second equation y = (5) + 2

2) So love for Y y = 5+ 2 = 7

Example

Substitution Example Two Line 1 : y=2x+1 Line 2 : 2y=3x-2

Step 1: Substitute one equation into the other 2(2x+1)=3x-2

Step 2: Now that you have a single variable equation, solve for that variable's equation 4x+2 = 3x-2 x+2= -2 x= –4

Step 3 : Once you have solved for the one variable insert that variable back into either equation to obtain the value of y at the solution. Insert x= –4 to find y value y = 2(–4)+1= –7

This example's solution is ( –4, –7).

Elimination Method

Elimination method is an algebraic method for solving systems. To use elimination you perform an operation on 1 equation then add the two equations so that one of the variables cancels.

Example of Elimination Line 1: y = x + 1 Line 2: y = –x

Elimination Method

Matrix Methods of SolutionCramer's rule is an explicit formula for the solution of a system of linear equations, with each variable given by a quotient of two determinants. For example, the solution to the system

is given by

Inverse Method

The Matrix form of a linear system of equations is

Ax= b If the coefficient matrix is non-

singular then the solution is: x = A-1b

Elementary Row Transformation

Solving a system of linear equations by reducing the augmented matrix of the system to row canonical form

Example

Example. Solve the system

The augmented matrix is

Example

Homogeneous System

A system of linear equations is homogeneous if all of the constant terms are zero:

A homogeneous system is equivalent to a matrix equation of the form

where A is an m × n matrix, x is a column vector with n entries, and 0 is the zero vector with m entries.

Homogeneous System

Homogeneous Systems are always ConsistentEvery homogeneous system has at least one

solution, known as the zero solution (or trivial solution), which is obtained by assigning the value of zero to each of the variables. The solution set has the following additional properties:

If u and v are two vectors representing solutions to a homogeneous system, then the vector sum u + v is also a solution to the system.

If u is a vector representing a solution to a homogeneous system, and r is any scalar, then ru is also a solution to the system.

Homogeneous System

Suppose that a homogeneous system of linear equations has m equations and n variables with n>m. Then the system has infinitely many solutions.

Example

Example

Example

Generalized Inverse

Generalized Inverse

Generalized Inverse

Method of Obtaining g-Inverse

Example

Method of Obtaining g- Inverse

Example

Properties of g Inverse of X’X

The matrix X’X has an important role in statistics where it arises in least square equations X’Xb = X’y. When G is a generalized inverse of X’X:

1.G’ is also a generalized inverse of X’X2.XGX’X=X, i.e., GX’ is a generalized inverse of X3.XGX’ is invariant to G4.XGX’ is symmetric whether G is or not.Let X+ be the Moore – Penrose inverse of X. ThenXX+ = XGX’ but X+ may not be equal to GX’.

g-Inverse

g-Inverse

Properties

Every singular symmetric matrix (of order two or more) has both symmetric and non-symmetric generalized inverses.

Let A represent a matrix of full column rank and B a matrix of

full row rank. Then, (1) a matrix G is a generalized inverse of A if and only if G

is a left inverse of A. And, (2) a matrix G is a generalized inverse of B if and only

if G is a right inverse of B.

Properties

For any matrix A and any nonzero scalar k, (1/k)A− is a generalized inverse of kA.

For any matrix A,−A− is a generalized inverse of −A.

For any matrix A, (A−)’ is a generalized inverse of A’ .

For any symmetric matrix A, (A−)’ is a generalized inverse of A’ .

Properties

A linear system Ax = b is consistent if and only if

AA- b = bor, equivalently, if and only if(I − AA-)B = 0

The Moore-Penrose Inverse

Given any matrix A, there is a unique matrix M such that

(i) AMA=A (ii) MAM=M (iii) AM is symmetric (iv) MA is symmetricThen matrix M is called the Moore-Penrose

inverse of A.M = L’(K’AL’)-1 K’

The Moore-Penrose Inverse

Example

Example