View
214
Download
0
Category
Preview:
Citation preview
CSC5160 Topics in Algorithms
Tutorial 1
Jan 25 2007Jerry Le
jlle@cse.cuhk.edu.hk
Outline
• Basic Linear Algebra – a quick review Vector space, Spanning set Linear independency Matrix: operations, rank, inverse,
eigenvalue, determinant
• Introduction to Linear Programming
Linear AlgebraVector space, spanning set
• Vector space- a set V that is closed under the ‘+’ and ‘•’ operations
- e.g.
VBV and ABAV, Then If A,B
R
RR
2102
210
32
,,
0,,
aaaxaxaaP
zyx
z
y
x
P ,
Note: A vector space does not necessarily contain “vectors”!
Linear AlgebraVector space, spanning set
• Spanning set- Definition: the spanning set of a nonempty set S is the set of all linear combinations of vectors in S
- Definition: a subspace of a vector space is itself a vector space
- Lemma: The span of a subspace is itself a vector space
- e.g.
SssccscscSspan nnnn
,,,,)( 1111 and R
2
1
1,
1
1R is span its , set consider
Linear AlgebraLinear independency
• Linear independency- Definition: A subset of a vector space is linearly independent if none of its elements can be represented as a linear combination of the others.
- e.g.
2
2,
1
1,
1
1
linearly independent
linearly dependent
Linear AlgebraLinear independency
• Linear independency- in other words:
- Basis
Definition: A basis of vector space V is a linearly independent subset whose span is V
ia
VaVaaaa
VVV
i
nnn
n
,0
0,,,
,,,
1121
21
are
satisfy that only the iff
tindependenlinearly are vectors The
R
22
2
2,
1
1,
1
1RR of basis a Not is it but , spans
Linear AlgebraMatrix
• Operations
- sums and scalar products
- multiplication
• Rank of a matrixDefinition: the rank of a matrix A is the maximal number of linearly independent rows (or columns) of A
?
333231
232221
131211
232221
131211
bbb
bbb
bbb
aaa
aaa
Linear AlgebraMatrix
• Rank of a matrix (cnt.)- Properties (consider an m-by-n matrix A) 1. the rank of A is at most min(m,n) 2. if the rank is n (m), A has full column (row) rank 3. if A is a square matrix (m=n), A is invertible iff it has rank n (Invertible: there exists B such that AB=In) 4. if B is an n-by-k matrix with rank n, then AB has the same rank as A 5. if B is an k-by-m matrix with rank m, then BA has the same rank as A …
Linear AlgebraMatrix
• Rank of a matrix (cnt.)- Computation
by Gauss elimination (or Gauss-Jordan reduction)
3212
11213
8112
1100
12/12/10
8112
1100
3010
2001rank: 3
Linear AlgebraMatrix
• Invertible matrix- an n-by-n matrix A is invertible iff
1. there exists an n-by-n matrix B such that AB=BA=In
OR 2. A has rank n
OR 3. the equation Ax=0 has only trivial solution x=0
OR 4. the equation Ax=b has exactly one solution in Rn
OR 5. the rows (columns) of A are linearly independent
OR 6. the rows (columns) of A form a basis of Rn
OR 5. the transpose AT is invertible
OR …
Linear AlgebraMatrix
• Eigenvalue & Eigenvector- consider an m-by-n matrix A as linear transformation:
- Definition: for an n-by-n matrix A, if there exist an non-zero vector x and a scalar λ, such that
Ax=λx
then λ is the eigenvalue of A, and x is the eigenvector of A.
- The eigenvalue (λ) “visualizes” the effect that a linear transformation (A) produces on a vector (x).
yAxxFyxF )(: mn RR
Linear AlgebraMatrix
• Eigenvalue & Eigenvector (cnt.)- how to compute?
Ax=λx Ax=λInx (A-λIn)x=0
(A-λIn)x=0 must have non-trivial solutions
i.e. B=(A-λIn)x is NOT invertible
• Determinant- how to determine if an (n-by-n) matrix (A) is invertible?
by Gauss elimination
Linear AlgebraMatrix
• Determinant (cnt.)- Gauss elimination: by linear row (column) operations
If AFJMX=0, then A is not invertibleso, can we define some function det(A), which is equal to AFJMX, or at least equal to λAFJMX?
'
0000
000
00
0
A
X
NM
LKJ
IHGF
EDCBA
yxwvu
tsrqp
onmlk
jihgf
edcba
A ji
i
ji
operations row/column
Linear AlgebraMatrix
• Determinant (cnt.)- Tracing every step of a specific process of Gauss elimination, we get
nspermutationn PaaaA
ihg
fed
cba
AegdhcfgdibfheiaA
dc
baAbcadA
)(,)2(,2)1(,1)det(
),()()()det(
,)det(
Generally,
Linear AlgebraMatrix
• Determinant (cnt.)- determinant as size functions
(x1,y1)
(x2,y2)
21
21
yy
xxA
det(A) is exactly the area of the parallelogram!
More generally, the absolute value of det(A) is the volume of the box formed by A’s column vectors V1,V2,…Vn in n-dimensional space
Outline
• Basic Linear Algebra – a quick review
• Introduction to Linear Programming Definition Example Basic feasible solution Geometry of LP
Linear programmingDefinition
• General optimization
• Linear programming
- when all f(x),gi(x),hj(x) are linear (general form)
nR
x
pjxh
mixg
xf
j
i
where
to subject
minimize
,,1,0)(
,,1,0)(
)(
inequality constraints
equality constraints
objective function
Linear programmingExample
• Example: the Diet problemFood (n kinds of food)
Nutrient … requirement
Fat
Vitamin aij ri
Protein
amount
boughtxj
Cost cj
m kin
ds o
f nu
trient
Linear programmingExample
• Example: the Diet problem- question: how to buy the food such that the cost is minimized, while the minimal nutrient requirement is satisfied?
0
min
x
rAx
cx
s.t.
nutrient th-i of trequiremen minimal r
j food of unit one of cost the :c
bought food th-j of amount x
food th-j of unit one in nutrient th-i of amount a
i
j
j
ij
:
:
:
Canonical form of LP (without equality constraints in the general form)
Linear programmingExample
• Example: the Diet problem- what if the amount of nutrient must be exactly the same with the requirement?
0
min
x
rAx
cx
s.t.
nutrient th-i of trequiremen r
j food of unit one of cost the :c
bought food th-j of amount x
food th-j of unit one in nutrient th-i of amount a
i
j
j
ij
:
:
:
Standard form of LP (without inequality constraints in the general form)
Linear programmingExample
• Equivalent of the three forms of LP- the general form, canonical form (without equality constraints) and standard form (without inequality constraints) are equivalent- One form can be transformed into another
Therefore, we can simply consider standard form without loss of generality for all LPs
0
is
bsAxbAx
bAx
bAxbAx
:constaintsequality into sconstraint inequality
:sconstraint inequality into sconstraintequality
Linear programmingBasic feasible solution
• Basic feasible solutions- consider the standard form of LP:
nR
cbxnmA
x
bAx
cx
,,
0
min
matrix, an is where
s.t.
How to find the solution?1. solve the equation system Ax=b (find basic solutions)2. pick the basic feasible solutions satisfying x≥0, out of the basic solutions3. find the solution that minimize cx
Linear programmingBasic feasible solution
• Basic feasible solutionshow to find the basic solution of Ax=b?• If A is invertible, then x has unique solution
• If A is not invertible, assuming A has rank m 1. select a collection of m linearly independent column vectors of A 2. set the components of x that are not corresponding to the selected column vectors to zero 3. solve the m resulting equations to get the remaining component of x
Linear programmingBasic feasible solution
• Basic feasible solutions - how to find the basic solution of Ax=b? (cnt.)
6
3
2
4
1000130
0100100
0010001
0001111
7
6
5
4
3
2
1
x
x
x
x
x
x
x
(0,0,0,4,2,3,6)feasible
6
3
2
4
1000130
0100100
0010001
0001111
7
6
5
4
3
2
1
x
x
x
x
x
x
x
(0,4,0,0,2,3,-6)not feasible (because x7<0)
Linear programmingGeometry of LP
• Geometry of LP- consider the LP
0,
12
max
yx
xy
yx
yx
s.t.
x
y
x+2y=1
y=x
feasible region
objective
Linear programmingGeometry of LP
• Geometry of LPin Rn space
- the linear constraints form a
closed convex polytope
- optimal solution appears on
the boundaries
- local optimal is global optimal
- Simplex algorithm: starting from a vertex, search along the edges until reaching the vertex of the optimal solution
Recommended