Upload
tranliem
View
215
Download
1
Embed Size (px)
Citation preview
Lecture 9: Vector Algebra
• Linear combination of vectors• Geometric interpretation
• Interpreting as Matrix-Vector Multiplication
• Span of a set of vectors
• Vector Spaces and Subspaces
• Linearly Independent/Dependent set of vectors and bases
• Spanning set
• Dimension
1
Linear Combination of Vectors
• Let 𝑣 =
𝑣1𝑣2𝑣3.𝑣𝑝
∈ 𝑅𝑝 and c1, c2, …, cp be scalars then vector y defined by:
is called a linear combination of v1, v2, …. vp with weights c1, c2, … cp.
• The weights in a linear combination can be any real numbers including zero.
• And ‘y’ is said to be linearly dependent on v1, v2, …. vp.
Linear Dependence
• Definition:
v1, v2, ……vn are linearly dependent if vi can be written down as a linearcombination of rest of the prceeding vectors for any i.
• A zero vector is linearly dependent on any set of N-dimensional vectors where 𝑥1, 𝑥2, … are all 0.
𝑣𝑖 = 𝑐1𝑣1 + 𝑐2𝑣2+ 𝑐3𝑣3+ …… . . 𝑐𝑖−1𝑣𝑖−1 𝑓𝑜𝑟 𝑎𝑛𝑦 𝑖 = 1,2… . . 𝑟
Linear Dependence
If there is a solution for the vector equation
In other words, there exists values for 𝑎1, 𝑎2, … 𝑎𝑛 such that the above equation is satisfied, then b is linearly dependent on 𝑥1, 𝑥2, … 𝑥𝑛.
Linear Independence
• If there is no value for 𝑎1, 𝑎2, … exists such that the below equation is satisfied then b is linearly independent of the set of vectors 𝒗𝟏, 𝒗𝟐, … 𝒗𝒏.
• In other words, 𝑥1𝒂𝟏 + 𝑥2𝒂𝟐 +⋯𝑥𝑛𝒂𝒏 ≠ 𝒃 for any value of 𝑥1, 𝑥2, …
or 𝑥1𝒂𝟏 + 𝑥2𝒂𝟐 +⋯𝑥𝑛𝒂𝒏 − 𝒃 ≠ 𝟎
means b is linearly independent of the set of vectors 𝒂𝟏, 𝒂𝟐, …𝒂𝒏
In general, the set of vectors {𝒂𝟏, 𝒂𝟐, …𝒂𝒏, 𝒃} is linearly independent if 𝑥1𝒂𝟏 + 𝑥2𝒂𝟐 +⋯𝑥𝑛𝒂𝒏 + 𝑥𝑛+1𝒃 = 𝟎, only if 𝑥1, 𝑥2, … are all 0.
Linear Independence
Linearly Independent/Dependent Set
• A set of vectors that are linearly independent is called a Linearly Independent Set.
• If at least one vector in a set of vectors is linearly dependent on other vectors, then that set is called a Linearly Dependent Set.
• A set of vectors with a zero vector has to be a Linearly Dependent Set.
.
Examples of Linear Dependence
• A vector and its scalar multiple are linear dependent. (By definition.) 𝑥1𝒂 = 𝒃
• A vector that is a linear combination of a set of vectors is linearly dependent on those vectors. (By definition.)
8
Examples of Linear Dependence
𝐿𝑒𝑡 𝑥1 = 1 2 3 , 𝑥2 = 2 4 5 𝑎𝑛𝑑 𝑏 = [4 8 11]
Then the rank of the matrix M is
𝑀 =1 2 32 4 54 8 11
- the number of linearly independent vectors
Span of a set of vectors
A vector that is in the span of a set of vectors is linearly dependent on those vectors. (By definition.)
Span of a set of vectors
• Example 1:
Span of a set of vectors
• Example 2:
𝑃1
𝒗𝟐
𝒗𝟏
𝑃2
0
Same R2 plane can be spanned by other vectors too…
• Let a=21
and b=12
.
• Any point in R2 can be represented
as a linear combination of the a and b axes.
• 𝑃𝑎𝑟𝑏𝑖𝑡𝑟𝑎𝑟𝑦 = 𝑐1𝒂 + 𝑐2𝒃
• Example:
• 𝑃1 =20
=𝟒
𝟑𝒂 +
−2
3𝒃
• 𝑃2 =−12
= −4
3𝒂 +
5
3𝒃
𝒂 𝑎𝑛𝑑 𝒃 vectors SPAN the R2 plane.
13
𝑃1
𝒃
𝒂
𝑃2𝟒
𝟑𝒂
−2
3𝒃
−𝟒
𝟑𝒂
5
3𝒃
a-coordinate b-coordinate
Span of a set of vectors (geometric description)A GEOMETRIC DESCRIPTION OF SPAN V
Let v be a nonzero vector in R3 . Then Span v is the set of all scalar
mult iples of v, which is the set of points on the line in R3 through v
and 0. See the figure below
Xiaohui Xie (UCI) ICS 6N January 17, 2017 17 / 18
Span of a set of vectors (geometric description)A GEOMETRIC DESCRIPTION OF SPAN U, V
If u and v are nonzero vectors in R3 , with v not a multiple of u, then
Span u, v is the plane in R3 that contains u, v, and 0.
In particular, Span u, v contains the line in R3 through u and 0 and
the line through v and 0. See the figure below.
Xiaohui Xie (UCI) ICS 6N January 17, 2017 18 / 18
How many minimum number of vectors are necessary to span a space? • ONE non-zero vector is required and sufficient to span a 1D “space” (in other
words, a line). The dimension of the vectors has to be 1 or more.
• TWO linearly independent vectors are required and sufficient to span a 2D plane such as the XY plane. The dimension of the vectors has to be 2 or more.
• THREE linearly independent vectors are required and sufficient to span a 3D space (such as XYZ volume). The dimension of the vectors has to be 3 or more.
• N linearly independent vectors are required and sufficient to span an N-D space. The dimension of the vectors has to be N or more.
• The span of these vectors is called the “vector space” (or “subspace”, if it is a subset of a vector space).
• (Later we will see that a vector space is more generic than a span. But in this course, we will only see vector spaces that are spans of a sets of vectors.)
16
Span of a set of vectors
• Maximal subset consisting of linearly independent vectors is called the basis of the span of x1, x2, x3, …… xn
• And the number of elements in the basis is called the dimension of the span.
Spanning Set
• A basis is an efficient spanning set that contains no unnecessary vectors.
• A basis can be constructed from a spanning set by discarding unneeded vectors
• Basis construction
Spanning Set
𝒗4=𝟐𝒗1 − 𝒗2. So, by Spanning Set Theorem, {𝒗1, 𝒗2, 𝒗3} span the same subspace as {𝒗1, 𝒗2, 𝒗3, 𝒗4}. We also see that {𝒗1, 𝒗2, 𝒗3} are linearly independent. So that is the basis of W.
Vector Spaces
• Span of vectors also form a vector space.
• Now we will look more into the definition of vector spaces and their properties.
Vector Spaces
7) 𝑐 𝑢 + 𝑣 = 𝑐𝑢 + 𝑐𝑣 for any scalar c
Vector Spaces
Vector Subspace
• If a subset of a vector space also forms a vector space, then that subset is called a vector subspace.
• Hence, every vector subspace is also a vector space. And also every vector space is also a vector subspace (of itself and possibly of larger spaces).
Vector Subspace
Vector Subspace
• Example
Vector Subspace
• Example
Vector Basis
Vector Basis
Example:
• No, because the Span{𝒗1, 𝒗2} ≠ 𝐻. • Span{𝒗1, 𝒗2} is the entire XY plane, not just the vectors of the form [s,s,0].
Spanning Set
• The following three sets in R3 show how linearly independent set can be enlarged to a basis and how further enlargement destroys the linear independence of the set.
Dimension of a Vector Space
• We already discussed • Dimension of a vector: Number of components of the vector• Dimension of a vector space (or subspace): Minimum number of linearly independent
vectors required to span that space.
• New definition• Dimension of a vector space (or subspace): Number of vectors in its basis.
• What is Basis of vector space again? • A Linearly Independent Set (of vectors) whose span is the vector space.
• This implies:• Basis has the minimum number of linearly independent vectors that spans the space.
• Note: Basis is not unique. But all bases that span the same space has the same number of vectors.
30
• Solving a system of 𝑚 equation and 𝑛 unknown in Algebra can be converted in matrix-vector multiplication form.
𝑎11𝑥1 + 𝑎12𝑥2 + 𝑎13𝑥3 = 𝑏1𝑎21𝑥1 + 𝑎22𝑥2 + 𝑎23𝑥3 = 𝑏2𝑎31𝑥1 + 𝑎32𝑥2 + 𝑎33𝑥3 = 𝑏3
• 𝐴 is a 3 × 3 matrix, 𝒙 and 𝒃 are 3 × 1 column vectors. 𝐴 and 𝒃 are known, where 𝒙 is unknown. This is a matrix equation.
𝐴𝒙 = 𝒃,
𝑎11 𝑎12 𝑎13𝑎21 𝑎22 𝑎23𝑎31 𝑎32 𝑎33
𝑥1𝑥2𝑥3
=
𝑏1𝑏2𝑏3
Matrix equation
Matrix equation as a vector equation
• Matrix equation can be interpreted as a vector equation.• The left side of equation (𝐴𝒙) can be written as:
𝑎11 𝑎12 𝑎13𝑎21 𝑎22 𝑎23𝑎31 𝑎32 𝑎33
𝑥1𝑥2𝑥3
=
𝑎11𝑥1 + 𝑎12𝑥2 + 𝑎13𝑥3𝑎21𝑥1 + 𝑎22𝑥2 + 𝑎23𝑥3𝑎31𝑥1 + 𝑎32𝑥2 + 𝑎33𝑥3
= 𝑥1
𝑎11𝑎21𝑎31
+ 𝑥2
𝑎12𝑎22𝑎32
+ 𝑥3
𝑎13𝑎23𝑎33
𝒄𝒐𝒍1 𝒄𝒐𝒍2 𝒄𝒐𝒍3 𝑥1𝒄𝒐𝒍1 𝑥2𝒄𝒐𝒍2 𝑥3𝒄𝒐𝒍3
Represented as a linear combination of column vectors
Matrix equation as a vector equation
• Now we have a vector equation, stating the same problem as 𝐴𝒙 = 𝒃
𝑥1
𝑎11𝑎21𝑎31
+ 𝑥2
𝑎12𝑎22𝑎32
+ 𝑥3
𝑎13𝑎23𝑎33
=
𝑏1𝑏2𝑏3
• So, we can represent any 𝐴𝒙 = 𝒃 equation as a vector equation.
• If 𝒃 is in the span of column vectors, the system has either ONE or INFINTEsolution.
• If 𝒃 is NOT in the span of column vectors, the system has NO solution. But, we can find projection of 𝒃 on the vector space, as best estimate.
Matrix equation (ONE solution)
• Example:
• 2𝑥 + 𝑦 = 5
4𝑥 − 2𝑦 = −2
• Row representation of above equation:
•2 14 −2
𝑥𝑦 =
5−2
13
𝑥
𝑦
Vector equation (ONE solution)
• Vector equation representation
of previous example:
• 𝑥24+ 𝑦
1−2
=5−2
• b is in the span of col1 and
col2.
• The only solution for this
equation is 𝑥 = 1, and 𝑦 = 3
dim1
dim2 𝒄𝒐𝒍1
𝒄𝒐𝒍2𝒃
Span of col1
Span of col2
Matrix equation (INFINITE solution)
• Example:
• 2𝑥 + 𝑦 = 34𝑥 + 2𝑦 = 6
• Row representation of above equation:
•2 14 2
𝑥𝑦 =
36
• Both equations are the same line
• All points on the line are solutions
𝑥
𝑦
Vector equation (INFINITE solution)
• Vector equation representation of previous example:
• 𝑥24+ 𝑦
12
=36
• 𝒄𝒐𝒍1 and 𝒄𝒐𝒍2 are linearly dependent• 𝒃 is in the span of 𝒄𝒐𝒍1 and 𝒄𝒐𝒍2
• 2𝑥 + 𝑦12
=36
• Any combination of 𝑥 and 𝑦 thatsatisfies 2𝑥 + 𝑦 = 3 is a solution.
dim1
dim2
𝒄𝒐𝒍1
𝒄𝒐𝒍2
𝒃
Span of col1 and Span of col2
Matrix equation (NO solution)
• Example:
• 2𝑥 + 𝑦 = 34𝑥 + 2𝑦 = 4
• Row representation of above equation:
•2 14 2
𝑥𝑦 =
34
• The lines are parallel.
• There is no solution.
𝑥
𝑦
Vector equation (NO solution)
• Vector equation representation
of previous example:
• 𝑥24+ 𝑦
12
=34
• 𝒄𝒐𝒍1 and 𝒄𝒐𝒍2 are linearly dependent
• 𝒃 is NOT in the span of 𝒄𝒐𝒍1 and 𝒄𝒐𝒍2• There is no solution
dim1
dim2
𝒄𝒐𝒍1
𝒄𝒐𝒍2
𝒃
Span of col1 and Span of col2
Vector equation (NO solution)
Hence, in order to solve thiswe project b onto the span of col1 and col2 using Least Squares
dim1
dim2
𝒄𝒐𝒍1
𝒄𝒐𝒍2
𝒃
Projection of b