Upload
hoangkhuong
View
219
Download
1
Embed Size (px)
Citation preview
Black Box Linear AlgebraAn Introduction to Wiedemann’s Approach
William J. Turner
Department of Mathematics & Computer Science
Wabash College
Overview
Symbolic computation
Black box matrix model
Wiedemann’s approach
Nonsingular systems
Singular systems
Rank
Determinant
Other work
Black Box Linear Algebra – p.1/32
Symbolic Computation
Superset of computer algebra
Different from numerical analysis
Symbols or exact arithmetic
Not floating point numbers (numerical analysis)
No round off errors
Algorithms may not be compatible
Black Box Linear Algebra – p.2/32
. . . vs. Numerical Analysis
Numerical Analysis
Find approximation quickly
May never find exact solution
Symbolic Computation
Find exact solution quickly
May never approximate
Black Box Linear Algebra – p.3/32
Probabilistic Algorithms
Three types:
Monte Carlo: Always fast, probably correct
Las Vegas: Probably fast, always correct
Requires certificate
BPP: Bounded Probabalistic Polynomial Time
Probably fast, probably correct
Atlantic City?
Schwartz-Zippel Lemma
Probability randomly choose root of polynomial
Black Box Linear Algebra – p.4/32
Solving Linear Systems
������������
� � � �
� � � �
� � �
� � � �
����������
� ��
�����������
�
� �
�����������
Black Box Linear Algebra – p.5/32
Gaussian Elimination
������������
� � � �
� � � �
� � �
� � � �
����������
� ��
�����������
�
�
�����������
� � � ��
�����������
�
� �
�����������
Row echelon form and back substitution
Must know how matrix stored
Matrix changed in calculation
Sparse matrices may become dense
Black Box Linear Algebra – p.6/32
Black Box Matrix Model
� � �� � � � � � ��
� � �� � �
External view of matrix
Only matrix-vector products allowed
Independent of implementation
Implementations efficient in time or space
Not unique to symbolic computation
Can compute Krylov sequence
� � � � � �
Black Box Linear Algebra – p.7/32
Examples
Matrix storage time
Arbitrary matrix � � � � � �
Sparse matrix
( � nonzero entries)
� � � � � �
Hilbert matrix
(
� � ��� � � �
� ��� )
� � � � � �
Toeplitz matrix
� � � � � � �� � � � �� � �� � � �
Black Box Linear Algebra – p.8/32
Linearly Recurrent Sequences
Let
�
be a vector space over field
�
.
A sequence
� � � � �
� � � � �is linearly recurrent if and only if there exist � � �
and
����� � �� � �� �� � � �
such that for all
� �
��� �
� � � � � � �
�
Black Box Linear Algebra – p.9/32
Linearly Recurrent Sequences
Let
�
be a vector space over field
�
.
A sequence
� � � � �
� � � � �is linearly recurrent if and only if there exist � � �
and
����� � �� � �� �� � � �
such that for all
� �� � �
���
� �� �
� � � � �
Black Box Linear Algebra – p.9/32
Generating Polynomials
The polynomial
� �� � �� � � � � �
generates �
� � � �
�� �
� � � � �� �
� � � � � � � � � �
� �
is an
� �� �
-module
Black Box Linear Algebra – p.10/32
Minimal Generating Polynomial
If
� � � � � and � � � �� �
, then
��
� � � � � � � � � � � � � � � �
� � � � �� � � � � � � � �
is an ideal
� �� �
is a principal ideal domain
There exists a unique monic generator of minimal degree, the
minimal generating polynomial of sequence
Minimal polynomial divides all generating polynomials
Black Box Linear Algebra – p.11/32
Example: Fibonacci Numbers
� � �
� � � � �� �� �� � �� �� �� � �� �� � ��
Minimal polynomial is
� � � � � �
� � � � � � � � �
�� � � � � � �
� � also generates �
� � � � � � � � �
� � � � � � � � � � �
also generates �
� � � � � � � � � � � �
Skips first
�
elements of �
Black Box Linear Algebra – p.12/32
Matrix Power Sequence
� � � � � � � � � � � � � �
��� � ��� �
generates
� � � � �
Cayley-Hamilton Theorem
Let
� �
be minimal polynomial of
� � � � �
� � � �� � ��� �
Black Box Linear Algebra – p.13/32
Krylov Sequence
� � � � � � � � � � � � � � � �
� � � � � � � � � � � � �
� � � � � � � � � � � � ��� � � � � � � � � � � � � � �
Let
� �� �
be minimal polynomial of
� � � � � �
� �� � � � � � �� � ��� �
Black Box Linear Algebra – p.14/32
Solving �
Suppose � �
��� �� � � �
, �� � � �� � � �, and � � � � ���
� �
If � exists, then
� �� �
satisfies requirements
� � � � � � � ��� � � � �� � � � �
� � �
� �
�� � � � � � � �
� �
��� � � � � ��� �
� � �
� �
��� � � � � ��� �
is a solution
Black Box Linear Algebra – p.15/32
Nonsingular
��� � � � � � �
�� � ��� � ��� � � � � �
� �� � � � � � �
� � �
� �
��� � � � � ��� �
is the unique solution
Black Box Linear Algebra – p.16/32
Bilinear Projection Sequence
� � ��T
� � � � � � � �T
� � � � � �
� � � � � � � � �T
� � � � � �
� � � � � � � � � � � � � �� � � � � � �
�T
� � � � � � � �
Let
� �� �� be minimal polynomial of
��T
� � � � �
� �� ��
� � �� � � � � � ��� � ��� � If � chosen randomly from finite
� � � � �
, then
� �� ��
� � �� �
with
probability at least
� �� � � � �� �
� � �
Certificate:� �� �
�
� � � � � � � � �
Black Box Linear Algebra – p.17/32
Computing Minimal Polynomial
Must know
��� � � � �
Extended Euclidean Algorithm
� � �� � � � �� � � �
Inputs are
�� �� �
� ��
�� � � � � � � and
�� �� � � � �
Stop when
�� � � � � � � �� � � � ���
Minimal polynomial is reversal of � � ��
Berlekamp-Massey Algorithm
From coding theory
Interpolates elements of scalar sequence
Related to Extended Euclidean Algorithm
Black Box Linear Algebra – p.18/32
Berlekamp-Massey Algorithm
Input: Scalar sequence
� � �
� � � � �
with generator � with�� � ��
� �
Output: Minimal polynomial
�
of sequence
1:
� � � �
Initial guess
�
2: for � � � to
� � do
� �
generates � �� � ��� � �
3:
� � � � � � � � � � � � ��� � �4:
� � �� ��� � � � � ��� � � ��� � � � ��� � �
5: if
� � � � then
� �
does not generate � � � � � ��
6: update
�
to generate � � � � ���
7: end if
8: end for
Black Box Linear Algebra – p.19/32
Example: Fibonacci Numbers
� � �
� � � � �� �� �� � �� �� �� � �� �� � ��
� �
recursion relation
� � � � � �
� � � � � � � �
� � � � � � � � � �
� � � � � � � � � � �
� � � � � � � � � � �
� � � � � � � � � � �
� � � � � � � � � � �
� � � � � � � � � � �
Black Box Linear Algebra – p.20/32
Wiedemann’s Algorithm
� �� ��
� � �� � � � � � ��� � ��� �
��� � � � �� ��
� �� � � �� � ��� � � �
Only need
��T
� � � � � �
�� �
Require: nonsingular
� � �� � �
and
� � ��
Ensure: � � ��
such that
� � � �1:
� �
random vector in
� �
where�� �
2: use Berlekamp-Massey to compute
� �� �� �� � � � � � � � � � � �
Store only
� � � �
3:
� �
��� � � � � � �� �� � � � � � �
Black Box Linear Algebra – p.21/32
Back to original problem
�����������
� � � �
� � � �
� � �
� � � �
���������
� ��
����������
�
� �
����������
� � �
� ��
�����������
�
�
�����������
Black Box Linear Algebra – p.22/32
Back to original problem
��T � � � � � �
�� � � � �� �� �� �� � �� � �� � �� � � �
� �� �� � �� �
�
� � � � �
� ��
� �� � � ��
���������
������������
�
�����������
�
������������
�
� �
�����������
���������
��
�����������
�
� �
�����������
Black Box Linear Algebra – p.23/32
Singular
Kaltofen and Saunders (1991):
If
� � � �� � � � � � known
� � � leading principal minor nonzero
Randomly choose �
Solve � � � nonsingular system
�� � � �� � � � � �� � � �
�� is leading � � � submatrix of
�
� �
is first � rows of
�
Then � �� � � ��
�� � �� �
��
� uniformly samples solution manifold
Black Box Linear Algebra – p.24/32
Precondition Matrix
Want � � � leading principal minor of
� �
nonzero
� � � � �
or
� � � � �
Need efficient matrix-vector product
New linear system
� � � � ���
Black Box Linear Algebra – p.25/32
Generic Rank ProfileWiedemann (1986):
� � � � �
,
�
parameterized and can realize any permutation
Kaltofen and Saunders (1991):
� � � ��� � ��� ,
��� unit upper triangular Toeplitz and
��� unit lower
triangular Toeplitz
Chen et al. (2002); Turner (2002):
� � � � � � � � ,
� � ,
� � based on butterfly networks
0 1 0 0 1 1 10
0 0 0 01 1 1 1
0 1 101 0 0 1
0 001 1 1 01
Generic exchange matrix mixes inputs, moving linear independence
Turner (2002): Generalize butterfly networks to radix-
�
switches
Turner (2004): Toeplitz matrix switches
Students: Hankel matrix switches
Black Box Linear Algebra – p.26/32
Matrix Rank
Kaltofen and Saunders (1991):
If
� � � �� � � � � � (unknown)
Leading principal minors nonzero up to
��
� � �� � � �� � ��
�
Then � � �� � � � �� � with probability at least
� � � � �
� � �
Black Box Linear Algebra – p.27/32
Matrix Minimal Polynomial
If � chosen randomly, then
� �� � � � �
with probability at least
� �� � � � �
� � �
� �
�� � �
If � and � chosen randomly, then
� �� ��
� � �with probability at
least
� �� � � � �
� � �
� � �
� � �
Black Box Linear Algebra – p.28/32
Rank PreconditionersGeneric rank profile preconditioner and diagonal matrix
Chen et al. (2002):
� � � ��
� ���
���
�� and
��� unit lower triangular Toeplitz,
��� upper triangular
Toeplitz
Turner (2002, 2003): Relax slightly generic rank profile
Turner (2004):
� � � � �
�����
� . . .
��
����
�
Toeplitz
Turner (2004):� � � � �
Black Box Linear Algebra – p.29/32
Matrix Determinant
If
� �� ��� � ��� �
Then
�� � � � � � �
� � � � �
Black Box Linear Algebra – p.30/32
Determinant PreconditionersGeneric rank profile preconditioner and diagonal matrix
Kaltofen and Pan (1992):
� � � � � � �
� unit upper triangular Toeplitz and
� � lower triangular Toeplitz
Chen et al. (2002):
� � � ��
���
�
. . .�
��
��
Turner (2002, 2003):� � � �
�����������
� �
. . .. . .
� � � � �
����������
Black Box Linear Algebra – p.31/32
Other Work
Krylov methods
Sparse preconditioners for small fields
Blocked algorithms
Preconditioners with more structure
Black Box Linear Algebra – p.32/32
ReferencesE. R. BERLEKAMP (1968). Algebraic Coding Theory. McGraw-Hill, New York.
L. CHEN, W. EBERLY, E. KALTOFEN, B. D. SAUNDERS, W. J. TURNER, and G. VILLARD (2002). Efficient Matrix Preconditioners
for Black Box Linear Algebra. Linear Algebra and its Applications, 343-344: 119–146. Special issue on Infinite Systems
of Linear Equations Finitely Specified.
J. L. DORNSTETTER (1987). On the Equivalence Between Berlekamp’s and Euclid’s Algorithms. IEEE Transactions on
Information Theory, IT-33(3): 428–431.
J. VON ZUR GATHEN and J. GERHARD (1999). Modern Computer Algebra. Cambridge University Press, Cambridge.
E. KALTOFEN (1992). Efficient Solution of Sparse Linear Systems. Lecture notes, Computer Science Department, Rensselaer
Polytechnic Institute, Troy, N.Y.
E. KALTOFEN and V. PAN (1991). Processor Efficient Parallel Solution of Linear Systems over an Abstract Field. In Proceedings
of SPAA ’91 3rd Annual ACM Symposium on Parallel Algorithms and Architectures, 180–191. ACM Press, New York, New
York.
E. KALTOFEN and V. PAN (1992). Processor Efficient Parallel Solution of Linear Systems II: the Positive Characteristic and
Singular Cases. In Proceedings of the 33rd Annual Symposium on Foundations of Computer Science, 714–723. IEEE
Computer Society Press, Los Alamitos, California.
E. KALTOFEN and B. D. SAUNDERS (1991). On Wiedemann’s Method of Solving Sparse Linear Systems. In H. F. MATTSON,
T. MORA, and T. R. N. RAO (eds.), AAECC-9: Proceedings of the 1991 Applied Algebra, Algebraic Algorithms and Error-
Correcting Codes, International Conference, vol. 539 of Lecture Notes in Computer Science, 29–38. Springer Verlag,
Heidelberg, Germany.
J. L. MASSEY (1969). Shift-Register Synthesis and BCH Decoding. IEEE Transactions on Information Theory, IT-15(1):
122–127.
J. T. SCHWARTZ (1980). Fast Probabilistic Algorithms for Verification of Polynomial Identities. J. ACM, 27: 701–717.
W. J. TURNER (2002). Black Box Linear Algebra with the LinBox Library. Ph.D. thesis, North Carolina State University, Raleigh,
North Carolina.
W. J. TURNER (2003). Determinantal Divisors and Matrix Preconditioners. Submitted to Journal of Symbolic Computation.
W. J. TURNER (2004). Preconditioners for Singular Black Box Matrices. Work in progress.
D. H. WIEDEMANN (1986). Solving Sparse Linear Equations Over Finite Fields. IEEE Transactions on Information Theory,
IT-32(1): 54–62.
R. ZIPPEL (1979). Probabilistic Algorithms for Sparse Polynomials. In E. W. NG (ed.), Symbolic and Algebraic Computation,
EUROSAM ’79, An International Symposiumon Symbolic and Algebraic Computation, Marseille, France, June 1979,
Proceedings, vol. 72 of Lecture Notes in Computer Science, 216–226. Springer Verlag, Heidelberg, Germany.
R. ZIPPEL (1990). Interpolating Polynomials from Their Values. Journal of Symbolic Computation, 9(3): 375–403.
32-1