Upload
saif-taurean
View
233
Download
0
Embed Size (px)
Citation preview
8/8/2019 Gaussian Elimination..14
1/12
Topic: Gaussian elimination method and
Gauss-Jordan method
NAME:- Shrashti Sachdev
ROLL NO:- RE6002A14
ACKNOWLEDEMENT:-
I WOULD LIKE TO BE THANKFUL TO MY TEACHER WHO GAVE ME SUCH A
NICE TOPIC BY WHICH I ABLE TO LEARN THESE THINGS AND OFCOURSE TELLYOU ABOUT THESE THINGS . I WOULD ALSO LIKE TO APPRECIATE
INTERNET SERVICES WITH THE HELP OF WHICH I HAVE COLLECTED THIS
MATERIAL ESPECIALLY GOOGLE SERVICES AND WIKIPEDIA SERVICES.
AT THE LAST BUT NOT LEAST I M THANKFUL TO MY PARENTS WHO
BROUGHT ME TO THIS LEVEL ,MY FRIENDS WHO GUIDED ME AT EVERY
MOMENT AND MY TEACHERS WHO HELPED ME IN UPDATING MY
KNOWLEDGE.
8/8/2019 Gaussian Elimination..14
2/12
8/8/2019 Gaussian Elimination..14
3/12
INTRODUCTION
Gaussian elimination is an algorithm for solving
1.System of linear equation
2.rank of matrix
3.inverse of an invertible matrix
Gaussian elimination is named after German mathematician and scientist Carl FriedrichGauss, which makes it an example ofStigler's law.
Elementary row operations are used to reduce a matrix torow echelon form. GaussJordan elimination, an extension of this algorithm, reduces the matrix further to reduced
row echelon form. Gaussian elimination alone is sufficient for many applications, and is
cheaper than the -Jordan version.
History
The method of Gaussian elimination appears in Chapter Eight,Rectangular Arrays, of the
important Chinese mathematical textJiuzhang suanshu orThe Nine Chapters on the
Mathematical Art. Its use is illustrated in eighteen problems, with two to five equations.The first reference to the book by this title is dated to 179 CE, but parts of it were written
as early as approximately 150 BCE.[1]It was commented on by Liu Hui in the 3rd
century.
The method in Europe stems from the notes of Isaac Newton.[2] In 1670, he wrote that allthe algebra books known to him lacked a lesson for solving simultaneous equations,
which Newton then supplied. Cambridge University eventually published the notes as
Arithmetica Universalis in 1707 long after Newton left academic life. The notes werewidely imitated, which made (what is now called) Gaussian elimination a standard lesson
in algebra textbooks by the end of the 18th century. Carl Friedrich Gauss in 1810 devised
a notation for symmetric elimination that was adopted in the 19th century by professional
hand computers to solve the normal equations of least-squares problems. The algorithmthat is taught in high school was named for Gauss only in the 1950s as a result of
confusion over the history of the subject.
http://en.wikipedia.org/wiki/Algorithmhttp://en.wikipedia.org/wiki/Carl_Friedrich_Gausshttp://en.wikipedia.org/wiki/Carl_Friedrich_Gausshttp://en.wikipedia.org/wiki/Stigler's_lawhttp://en.wikipedia.org/wiki/Stigler's_lawhttp://en.wikipedia.org/wiki/Elementary_row_operationshttp://en.wikipedia.org/wiki/Row_echelon_formhttp://en.wikipedia.org/wiki/Row_echelon_formhttp://en.wikipedia.org/wiki/Gauss%E2%80%93Jordan_eliminationhttp://en.wikipedia.org/wiki/Gauss%E2%80%93Jordan_eliminationhttp://en.wikipedia.org/wiki/Gauss%E2%80%93Jordan_eliminationhttp://en.wikipedia.org/wiki/The_Nine_Chapters_on_the_Mathematical_Arthttp://en.wikipedia.org/wiki/The_Nine_Chapters_on_the_Mathematical_Arthttp://en.wikipedia.org/wiki/Gaussian_elimination#cite_note-0http://en.wikipedia.org/wiki/Gaussian_elimination#cite_note-0http://en.wikipedia.org/wiki/Liu_Huihttp://en.wikipedia.org/wiki/Gaussian_elimination#cite_note-1http://en.wikipedia.org/wiki/Gaussian_elimination#cite_note-1http://en.wikipedia.org/wiki/Carl_Friedrich_Gausshttp://en.wikipedia.org/wiki/Carl_Friedrich_Gausshttp://en.wikipedia.org/wiki/Stigler's_lawhttp://en.wikipedia.org/wiki/Elementary_row_operationshttp://en.wikipedia.org/wiki/Row_echelon_formhttp://en.wikipedia.org/wiki/Gauss%E2%80%93Jordan_eliminationhttp://en.wikipedia.org/wiki/Gauss%E2%80%93Jordan_eliminationhttp://en.wikipedia.org/wiki/The_Nine_Chapters_on_the_Mathematical_Arthttp://en.wikipedia.org/wiki/The_Nine_Chapters_on_the_Mathematical_Arthttp://en.wikipedia.org/wiki/Gaussian_elimination#cite_note-0http://en.wikipedia.org/wiki/Liu_Huihttp://en.wikipedia.org/wiki/Gaussian_elimination#cite_note-1http://en.wikipedia.org/wiki/Algorithm8/8/2019 Gaussian Elimination..14
4/12
Gauss-Jordan Method
Definitions:
The Gauss-Jordan Elimination Method is a technique for solving systems of linearequations of any size.
An equivalent stage is each stage of a sequence of operations on a system of linearequations.
A matrix is a rectangular array of numbers.
A coefficient matrix is the matrix of only the coefficients of an equation.
An augmented matrix is the coefficient matrix augmented by the right-hand side.
An elementary row operation changes the form of a matrix but preserves the solutionof the original system.
An equivalent augmented matrix represents an equivalent system.
An example of how to use the Gauss-Jordan method;
The first linear equation.2x + 4y = 8 or; 2 4 8
The second linear equation.3x - 2y = 4 or; 3 2 4
The coefficient matrix consists of the coefficients of the first two columns.2x + 4y = 8
3x - 2y = 4
If you drop the x and y variables, you get the augmented matrix of the system.
2 4 8
3 -2 4
Procedure in the equation and matrix forms:
1. Set up the equations like this:
2x + 4y = 8 or; 2 4 8
3x - 2y = 4 or; 3 -2 4
2. Multiply the first equation by 1/2:
x + 2y = 4 or; 1 2 4
3x - 2y = 4 or; 3 -2 4
8/8/2019 Gaussian Elimination..14
5/12
3.A) Multiply the first equation by -3 and add the two equations:
-3x - 6y = -12 or; -3 -6 -12
3x - 2y = 4 or; 3 -2 4
0x -8y = -8 or; 0 -8 -8B) You get:
x + 2y = 4 or; 1 2 4
-8y = -8 or; 0 -8 -8
C) Multiply the second equation by -1/8:
x + 2y = 4 or; 1 2 4
y = 1 or; 0 1 1
4.A) Multiply the second equation by -2 and add the two equations:
x + 2y = 4 or; 1 2 4
-2y = -2 or; 0 -2 -2
x = 2 or; 0 1 2
B) You get:
x = 2 or; 1 0 2
y = 1 or; 0 1 1
5. The following is the solution of the system:
x = 2 or; 1 0 2
y = 1 or; 0 1 1
Algorithm overview
The process of Gaussian elimination has two parts. The first part (Forward Elimination)
reduces a given system to eithertriangularorechelon form, or results in adegenerate
equation with no solution, indicating the system has no solution. This is accomplished
through the use ofelementary row operations. The second step usesback substitution tofind the solution of the system above.
Stated equivalently for matrices, the first part reduces a matrix to row echelon form using
elementary row operations while the second reduces it to reduced row echelon form, orrow canonical form.
Another point of view, which turns out to be very useful to analyze the algorithm, is that
Gaussian elimination computes amatrix decomposition. The three elementary row
http://en.wikipedia.org/wiki/Triangular_formhttp://en.wikipedia.org/wiki/Echelon_formhttp://en.wikipedia.org/wiki/Degeneracy_(mathematics)http://en.wikipedia.org/wiki/Degeneracy_(mathematics)http://en.wikipedia.org/wiki/Elementary_row_operationshttp://en.wikipedia.org/wiki/Elementary_row_operationshttp://en.wikipedia.org/wiki/Triangular_matrix#Forward_and_back_substitutionhttp://en.wikipedia.org/wiki/Triangular_matrix#Forward_and_back_substitutionhttp://en.wikipedia.org/wiki/Elementary_row_operationshttp://en.wikipedia.org/wiki/Row_canonical_formhttp://en.wikipedia.org/wiki/Matrix_decompositionhttp://en.wikipedia.org/wiki/Matrix_decompositionhttp://en.wikipedia.org/wiki/Triangular_formhttp://en.wikipedia.org/wiki/Echelon_formhttp://en.wikipedia.org/wiki/Degeneracy_(mathematics)http://en.wikipedia.org/wiki/Elementary_row_operationshttp://en.wikipedia.org/wiki/Triangular_matrix#Forward_and_back_substitutionhttp://en.wikipedia.org/wiki/Elementary_row_operationshttp://en.wikipedia.org/wiki/Row_canonical_formhttp://en.wikipedia.org/wiki/Matrix_decomposition8/8/2019 Gaussian Elimination..14
6/12
operations used in the Gaussian elimination (multiplying rows, switching rows, and
adding multiples of rows to other rows) amount to multiplying the original matrix with
invertible matrices from the left. The first part of the algorithm computes anLUdecomposition, while the second part writes the original matrix as the product of a
uniquely determined invertible matrix and a uniquely determined reduced row-echelon
matrix.
Example
Suppose the goal is to find and describe the solution(s), if any, of the followingsystem of
linear equations:
The algorithm is as follows: eliminatex from all equations belowL1, and then eliminatey
from all equations belowL2. This will put the system intotriangular form. Then, usingback-substitution, each unknown can be solved for.
In the example,x is eliminated fromL2 by adding toL2.x is then eliminated fromL3by addingL1 toL3. Formally:
The result is:
Nowy is eliminated fromL3 by adding 4L2 toL3:
The result is:
http://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/System_of_linear_equationshttp://en.wikipedia.org/wiki/System_of_linear_equationshttp://en.wikipedia.org/wiki/System_of_linear_equationshttp://en.wikipedia.org/wiki/Triangular_formhttp://en.wikipedia.org/wiki/Triangular_formhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/System_of_linear_equationshttp://en.wikipedia.org/wiki/System_of_linear_equationshttp://en.wikipedia.org/wiki/Triangular_form8/8/2019 Gaussian Elimination..14
7/12
This result is a system of linear equations in triangular form, and so the first part of the
algorithm is complete.
The second part, back-substitution, consists of solving for the unknowns in reverse order.
It can thus be seen that
Then,zcan be substituted intoL2, which can then be solved to obtain
Next,zandy can be substituted intoL1, which can be solved to obtain
The system is solved.
Some systems cannot be reduced to triangular form, yet still have at least one validsolution: for example, ify had not occurred inL2 andL3 after the first step above, the
algorithm would have been unable to reduce the system to triangular form. However, it
would still have reduced the system toechelon form. In this case, the system does nothave a unique solution, as it contains at least onefree variable. The solution set can thenbe expressed parametrically (that is, in terms of the free variables, so that if values for the
free variables are chosen, a solution will be generated).
In practice, one does not usually deal with the systems in terms of equations but insteadmakes use of the augmented matrix (which is also suitable for computer manipulations).
For example:
Therefore, the Gaussian Elimination algorithm applied to the augmented matrix begins
with:
http://en.wikipedia.org/wiki/Echelon_formhttp://en.wikipedia.org/wiki/Echelon_formhttp://en.wikipedia.org/wiki/Echelon_formhttp://en.wikipedia.org/wiki/Free_variablehttp://en.wikipedia.org/wiki/Free_variablehttp://en.wikipedia.org/wiki/Augmented_matrixhttp://en.wikipedia.org/wiki/Augmented_matrixhttp://en.wikipedia.org/wiki/Echelon_formhttp://en.wikipedia.org/wiki/Free_variablehttp://en.wikipedia.org/wiki/Augmented_matrixhttp://en.wikipedia.org/wiki/Augmented_matrix8/8/2019 Gaussian Elimination..14
8/12
which, at the end of the first part(Gaussian elimination, zeros only under the leading 1) ofthe algorithm, looks like this:
That is, it is inrow echelon form.
At the end of the algorithm, if the GaussJordan elimination(zeros under and above the
leading 1) is applied:
That is, it is inreduced row echelon form, or row canonical form.
Other applications
Finding the inverse of a matrix
SupposeA is a matrix and you need to calculate itsinverse. The identity
matrix is augmented to the right ofA, forming a matrix (theblock matrixB =
[A,I]). Through application of elementary row operations and the Gaussian elimination
algorithm, the left block ofB can be reduced to the identity matrixI, which leavesA 1 inthe right block ofB.
If the algorithm is unable to reduceA to triangular form, thenA is not invertible.
[edit] General algorithm to compute ranks and bases
The Gaussian elimination algorithm can be applied to any matrixA. If we get
"stuck" in a given column, we move to the next column. In this way, for example, some
matrices can be transformed to a matrix that has a reduced row echelon form like
http://en.wikipedia.org/wiki/Row_echelon_formhttp://en.wikipedia.org/wiki/Row_echelon_formhttp://en.wikipedia.org/wiki/Gauss%E2%80%93Jordan_eliminationhttp://en.wikipedia.org/wiki/Reduced_row_echelon_formhttp://en.wikipedia.org/wiki/Reduced_row_echelon_formhttp://en.wikipedia.org/wiki/Matrix_inversionhttp://en.wikipedia.org/wiki/Matrix_inversionhttp://en.wikipedia.org/wiki/Matrix_inversionhttp://en.wikipedia.org/wiki/Identity_matrixhttp://en.wikipedia.org/wiki/Identity_matrixhttp://en.wikipedia.org/wiki/Block_matrixhttp://en.wikipedia.org/w/index.php?title=Gaussian_elimination&action=edit§ion=6http://en.wikipedia.org/wiki/Row_echelon_formhttp://en.wikipedia.org/wiki/Gauss%E2%80%93Jordan_eliminationhttp://en.wikipedia.org/wiki/Reduced_row_echelon_formhttp://en.wikipedia.org/wiki/Matrix_inversionhttp://en.wikipedia.org/wiki/Identity_matrixhttp://en.wikipedia.org/wiki/Identity_matrixhttp://en.wikipedia.org/wiki/Block_matrixhttp://en.wikipedia.org/w/index.php?title=Gaussian_elimination&action=edit§ion=68/8/2019 Gaussian Elimination..14
9/12
(the *'s are arbitrary entries). This echelon matrix Tcontains a wealth of information
aboutA: therankofA is 5 since there are 5 non-zero rows in T; thevector spacespanned
by the columns ofA has a basis consisting of the first, third, fourth, seventh and ninthcolumn ofA (the columns of the ones in T), and the *'s tell you how the other columns of
A can be written as linear combinations of the basis columns.
Analysis
Gaussian elimination to solve a system ofn equations forn unknowns requires n(n+1) / 2
divisions, (2n3 + 3n2 5n)/6 multiplications, and (2n3 + 3n2 5n)/6 subtractions,[3] for a
total of approximately 2n3 / 3 operations. So it has a complexity of .
This algorithm can be used on a computer for systems with thousands of equations and
unknowns. However, the cost becomes prohibitive for systems with millions of equations.
These large systems are generally solved using iterative methods. Specific methods existfor systems whose coefficients follow a regular pattern (seesystem of linear equations).
The Gaussian elimination can be performed over anyfield.
Gaussian elimination is numerically stable fordiagonally dominant orpositive-definite
matrices. For general matrices, Gaussian elimination is usually considered to be stable in
practice if you usepartial pivoting as described below, even though there are examplesfor which it is unstable.[4]
Higher order tensors
Gaussian elimination does not generalize in any simple way to higher ordertensors
(matrices are order 2 tensors); even computing the rank of a tensor of order greater than 2
is a difficult problem.
Pseudocode
http://en.wikipedia.org/wiki/Rank_of_a_matrixhttp://en.wikipedia.org/wiki/Rank_of_a_matrixhttp://en.wikipedia.org/wiki/Vector_spacehttp://en.wikipedia.org/wiki/Vector_spacehttp://en.wikipedia.org/wiki/Vector_spacehttp://en.wikipedia.org/wiki/Gaussian_elimination#cite_note-2http://en.wikipedia.org/wiki/Iterative_methodhttp://en.wikipedia.org/wiki/System_of_linear_equationshttp://en.wikipedia.org/wiki/System_of_linear_equationshttp://en.wikipedia.org/wiki/Field_(mathematics)http://en.wikipedia.org/wiki/Field_(mathematics)http://en.wikipedia.org/wiki/Numerical_stabilityhttp://en.wikipedia.org/wiki/Diagonally_dominanthttp://en.wikipedia.org/wiki/Diagonally_dominanthttp://en.wikipedia.org/wiki/Positive-definite_matrixhttp://en.wikipedia.org/wiki/Positive-definite_matrixhttp://en.wikipedia.org/wiki/Pivot_elementhttp://en.wikipedia.org/wiki/Gaussian_elimination#cite_note-3http://en.wikipedia.org/wiki/Gaussian_elimination#cite_note-3http://en.wikipedia.org/wiki/Tensorshttp://en.wikipedia.org/wiki/Rank_of_a_matrixhttp://en.wikipedia.org/wiki/Vector_spacehttp://en.wikipedia.org/wiki/Gaussian_elimination#cite_note-2http://en.wikipedia.org/wiki/Iterative_methodhttp://en.wikipedia.org/wiki/System_of_linear_equationshttp://en.wikipedia.org/wiki/Field_(mathematics)http://en.wikipedia.org/wiki/Numerical_stabilityhttp://en.wikipedia.org/wiki/Diagonally_dominanthttp://en.wikipedia.org/wiki/Positive-definite_matrixhttp://en.wikipedia.org/wiki/Pivot_elementhttp://en.wikipedia.org/wiki/Gaussian_elimination#cite_note-3http://en.wikipedia.org/wiki/Tensors8/8/2019 Gaussian Elimination..14
10/12
As explained above, Gaussian elimination writes a given m n matrixA uniquely as a
product of an invertible m m matrix Sand a row-echelon matrix T. Here, Sis the
product of the matrices corresponding to the row operations performed.
The formal algorithm to compute TfromA follows. We writeA[i,j] for the entry in row i,
columnj in matrixA. The transformation is performed "in place", meaning that theoriginal matrixA is lost and successively replaced by T.
i := 1j := 1while (i m andj n) do Find pivot in column j, starting in row i:maxi := i
for k := i+1 to m do if abs(A[k,j]) > abs(A[maxi,j]) then
maxi := k end if end for
if
A[maxi,j] 0then
swap rows i and maxi, but do not change the value of i Now A[i,j] will contain the old value of A[maxi,j].
divide each entry in row i by A[i,j] Now A[i,j] will have the value 1. for u := i+1 to m do
subtract A[u,j] * row i from row u Now A[u,j] will be 0, since A[u,j] - A[i,j] * A[u,j] = A[u,j] - 1* A[u,j] = 0.
end fori := i + 1
end ifj := j + 1
end while
This algorithm differs slightly from the one discussed earlier, because before eliminating
a variable, it first exchanges rows to move the entry with the largest absolute value to the
"pivot position". Such "partial pivoting" improves the numerical stability of thealgorithm; some variants are also in use.
The column currently being transformed is called the pivot column. Proceed from left to
right, letting the pivot column be the first column, then the second column, etc. and
finally the last column before the vertical line. For each pivot column, do the followingtwo steps before moving on to the next pivot column:
1. Locate the diagonal element in the pivot column. This element is called the pivot.
The row containing the pivot is called the pivot row. Divide every element in the
pivot row by the pivot to get a new pivot row with a 1 in the pivot position.2. Get a 0 in each position below the pivot position by subtracting a suitable multiple
of the pivot row from each of the rows below it.
http://en.wikipedia.org/wiki/Absolute_valuehttp://en.wikipedia.org/wiki/Absolute_value8/8/2019 Gaussian Elimination..14
11/12
Upon completion of this procedure the augmented matrix will be inrow-echelon form
and may be solved by back-substitution.
With the increasing popularity of multi-core processors, programmers now exploit
thread-level parallel Gaussian elimination algorithms to increase the speed of computing.
The shared-memory programming model (as opposed to the message exchange model)pseudocode is listed below.
void parallel(int num_threads,int matrix_dimension)int i;for(i=0;icur_count=0;pthread_cond_broadcast(&(mybarrier->barrier_cond));
pthread_mutex_unlock(&(mybarrier->barrier_mutex));
http://en.wikipedia.org/wiki/Row-echelon_formhttp://en.wikipedia.org/wiki/Row-echelon_formhttp://en.wikipedia.org/wiki/Row-echelon_form8/8/2019 Gaussian Elimination..14
12/12
References
Atkinson, Kendall A. (1989), An Introduction to Numerical Analysis (2nd ed.),
New York:
Calinger, Ronald (1999), A Contextual History of Mathematics, Farebrother, R.W. (1988), Linear Least Squares Computations, STATISTICS:
Textbooks and Monographs, Marcel Dekker,
(1996), Matrix Computations (3rd ed.), Johns Hopkins,.
Katz, Victor J. (2004), A History of Mathematics, Brief Version