1SGEJSV(1)LAPACK routine (version 3.2) SGEJSV(1)
2
6 SGEJSV - [A], where M >= N
7
9 SUBROUTINE SGEJSV( JOBA, JOBU, JOBV, JOBR, JOBT, JOBP,
10 & M, N, A, LDA, SVA, U, LDU, V, LDV,
12 & WORK, LWORK, IWORK, INFO )
14 IMPLICIT NONE
16 INTEGER INFO, LDA, LDU, LDV, LWORK, M, N
18 REAL A( LDA, * ), SVA( N ), U( LDU, * ), V( LDV, * ),
20 & WORK( LWORK )
22 INTEGER IWORK( * )
24 CHARACTER*1 JOBA, JOBP, JOBR, JOBT, JOBU, JOBV
26
28 matrix [A], where M >= N. The SVD of [A] is written as
29 [A] = [U] * [SIGMA] * [V]^t,
30 where [SIGMA] is an N-by-N (M-by-N) matrix which is zero except for its
31 N diagonal elements, [U] is an M-by-N (or M-by-M) orthonormal matrix,
32 and [V] is an N-by-N orthogonal matrix. The diagonal elements of
33 [SIGMA] are the singular values of [A]. The columns of [U] and [V] are
34 the left and the right singular vectors of [A], respectively. The
35 matrices [U] and [V] are computed and stored in the arrays U and V,
36 respectively. The diagonal of [SIGMA] is computed and stored in the
37 array SVA.
38
40 JOBA (input) CHARACTER*1
41 Specifies the level of accuracy:
42 = 'C': This option works well (high relative accuracy) if A = B
43 * D, with well-conditioned B and arbitrary diagonal matrix D.
44 The accuracy cannot be spoiled by COLUMN scaling. The accuracy
45 of the computed output depends on the condition of B, and the
46 procedure aims at the best theoretical accuracy. The relative
47 error max_{i=1:N}|d sigma_i| / sigma_i is bounded by
48 f(M,N)*epsilon* cond(B), independent of D. The input matrix is
49 preprocessed with the QRF with column pivoting. This initial
50 preprocessing and preconditioning by a rank revealing QR factor‐
51 ization is common for all values of JOBA. Additional actions are
52 specified as follows:
53 = 'E': Computation as with 'C' with an additional estimate of
54 the condition number of B. It provides a realistic error bound.
55 = 'F': If A = D1 * C * D2 with ill-conditioned diagonal scalings
56 D1, D2, and well-conditioned matrix C, this option gives higher
57 accuracy than the 'C' option. If the structure of the input
58 matrix is not known, and relative accuracy is desirable, then
59 this option is advisable. The input matrix A is preprocessed
60 with QR factorization with FULL (row and column) pivoting. =
61 'G' Computation as with 'F' with an additional estimate of the
62 condition number of B, where A=D*B. If A has heavily weighted
63 rows, then using this condition number gives too pessimistic
64 error bound. = 'A': Small singular values are the noise and the
65 matrix is treated as numerically rank defficient. The error in
66 the computed singular values is bounded by f(m,n)*epsilon*||A||.
67 The computed SVD A = U * S * V^t restores A up to
68 f(m,n)*epsilon*||A||. This gives the procedure the licence to
69 discard (set to zero) all singular values below N*epsilon*||A||.
70 = 'R': Similar as in 'A'. Rank revealing property of the initial
71 QR factorization is used do reveal (using triangular factor) a
72 gap sigma_{r+1} < epsilon * sigma_r in which case the numerical
73 RANK is declared to be r. The SVD is computed with absolute
74 error bounds, but more accurately than with 'A'.
75 JOBU (input) CHARACTER*1
77 Specifies whether to compute the columns of U:
78 = 'U': N columns of U are returned in the array U.
79 = 'F': full set of M left sing. vectors is returned in the array
80 U.
81 = 'W': U may be used as workspace of length M*N. See the
82 description of U. = 'N': U is not computed.
83 JOBV (input) CHARACTER*1
85 Specifies whether to compute the matrix V:
86 = 'V': N columns of V are returned in the array V; Jacobi rota‐
87 tions are not explicitly accumulated. = 'J': N columns of V are
88 returned in the array V, but they are computed as the product of
89 Jacobi rotations. This option is allowed only if JOBU .NE. 'N',
90 i.e. in computing the full SVD. = 'W': V may be used as
91 workspace of length N*N. See the description of V. = 'N': V is
92 not computed.
93 JOBR (input) CHARACTER*1
95 Specifies the RANGE for the singular values. Issues the licence
96 to set to zero small positive singular values if they are out‐
97 side specified range. If A .NE. 0 is scaled so that the largest
98 singular value of c*A is around SQRT(BIG), BIG=SLAMCH('O'), then
99 JOBR issues the licence to kill columns of A whose norm in c*A
100 is less than SQRT(SFMIN) (for JOBR.EQ.'R'), or less than
101 SMALL=SFMIN/EPSLN, where SFMIN=SLAMCH('S'), EPSLN=SLAMCH('E').
102 = 'N': Do not kill small columns of c*A. This option assumes
103 that BLAS and QR factorizations and triangular solvers are
104 implemented to work in that range. If the condition of A is
105 greater than BIG, use SGESVJ. = 'R': RESTRICTED range for
106 sigma(c*A) is [SQRT(SFMIN), SQRT(BIG)] (roughly, as described
107 above). This option is recommended. ===========================
108 For computing the singular values in the FULL range [SFMIN,BIG]
109 use SGESVJ.
110 JOBT (input) CHARACTER*1
112 If the matrix is square then the procedure may determine to use
113 transposed A if A^t seems to be better with respect to conver‐
114 gence. If the matrix is not square, JOBT is ignored. This is
115 subject to changes in the future. The decision is based on two
116 values of entropy over the adjoint orbit of A^t * A. See the
117 descriptions of WORK(6) and WORK(7). = 'T': transpose if
118 entropy test indicates possibly faster convergence of Jacobi
119 process if A^t is taken as input. If A is replaced with A^t,
120 then the row pivoting is included automatically. = 'N': do not
121 speculate. This option can be used to compute only the singular
122 values, or the full SVD (U, SIGMA and V). For only one set of
123 singular vectors (U or V), the caller should provide both U and
124 V, as one of the matrices is used as workspace if the matrix A
125 is transposed. The implementer can easily remove this con‐
126 straint and make the code more complicated. See the descriptions
127 of U and V.
128 JOBP (input) CHARACTER*1
130 Issues the licence to introduce structured perturbations to
131 drown denormalized numbers. This licence should be active if the
132 denormals are poorly implemented, causing slow computation,
133 especially in cases of fast convergence (!). For details see
134 [1,2]. For the sake of simplicity, this perturbations are
135 included only when the full SVD or only the singular values are
136 requested. The implementer/user can easily add the perturbation
137 for the cases of computing one set of singular vectors. = 'P':
138 introduce perturbation
139 = 'N': do not perturb
140 M (input) INTEGER
142 The number of rows of the input matrix A. M >= 0.
143 N (input) INTEGER
145 The number of columns of the input matrix A. M >= N >= 0.
146 A (input/workspace) REAL array, dimension (LDA,N)
148 On entry, the M-by-N matrix A.
149 LDA (input) INTEGER
151 The leading dimension of the array A. LDA >= max(1,M).
152 SVA (workspace/output) REAL array, dimension (N)
154 On exit, - For WORK(1)/WORK(2) = ONE: The singular values of A.
155 During the computation SVA contains Euclidean column norms of
156 the iterated matrices in the array A. - For WORK(1) .NE.
157 WORK(2): The singular values of A are
158 (WORK(1)/WORK(2)) * SVA(1:N). This factored form is used if
159 sigma_max(A) overflows or if small singular values have been
160 saved from underflow by scaling the input matrix A. - If
161 JOBR='R' then some of the singular values may be returned as
162 exact zeros obtained by "set to zero" because they are below
163 the numerical rank threshold or are denormalized numbers.
164 U (workspace/output) REAL array, dimension ( LDU, N )
166 If JOBU = 'U', then U contains on exit the M-by-N matrix of the
167 left singular vectors. If JOBU = 'F', then U contains on exit
168 the M-by-M matrix of the left singular vectors, including an
169 ONB of the orthogonal complement of the Range(A). If JOBU =
170 'W' .AND. (JOBV.EQ.'V' .AND. JOBT.EQ.'T' .AND. M.EQ.N), then U
171 is used as workspace if the procedure replaces A with A^t. In
172 that case, [V] is computed in U as left singular vectors of A^t
173 and then copied back to the V array. This 'W' option is just a
174 reminder to the caller that in this case U is reserved as
175 workspace of length N*N. If JOBU = 'N' U is not referenced.
176 The leading dimension of the array U, LDU >= 1. IF JOBU =
177 'U' or 'F' or 'W', then LDU >= M.
178 V (workspace/output) REAL array, dimension ( LDV, N )
180 If JOBV = 'V', 'J' then V contains on exit the N-by-N matrix of
181 the right singular vectors; If JOBV = 'W', AND (JOBU.EQ.'U' AND
182 JOBT.EQ.'T' AND M.EQ.N), then V is used as workspace if the
183 pprocedure replaces A with A^t. In that case, [U] is computed
184 in V as right singular vectors of A^t and then copied back to
185 the U array. This 'W' option is just a reminder to the caller
186 that in this case V is reserved as workspace of length N*N. If
187 JOBV = 'N' V is not referenced.
188 LDV (input) INTEGER
190 The leading dimension of the array V, LDV >= 1. If JOBV = 'V'
191 or 'J' or 'W', then LDV >= N.
192 WORK (workspace/output) REAL array, dimension at least LWORK.
194 On exit, WORK(1) = SCALE = WORK(2) / WORK(1) is the scaling
195 factor such that SCALE*SVA(1:N) are the computed singular val‐
196 ues of A. (See the description of SVA().) WORK(2) = See the
197 description of WORK(1). WORK(3) = SCONDA is an estimate for
198 the condition number of column equilibrated A. (If JOBA .EQ.
199 'E' or 'G') SCONDA is an estimate of SQRT(||(R^t *
200 R)^(-1)||_1). It is computed using SPOCON. It holds N^(-1/4) *
201 SCONDA <= ||R^(-1)||_2 <= N^(1/4) * SCONDA where R is the tri‐
202 angular factor from the QRF of A. However, if R is truncated
203 and the numerical rank is determined to be strictly smaller
204 than N, SCONDA is returned as -1, thus indicating that the
205 smallest singular values might be lost. If full SVD is needed,
206 the following two condition numbers are useful for the analysis
207 of the algorithm. They are provied for a developer/implementer
208 who is familiar with the details of the method. WORK(4) = an
209 estimate of the scaled condition number of the triangular fac‐
210 tor in the first QR factorization. WORK(5) = an estimate of
211 the scaled condition number of the triangular factor in the
212 second QR factorization. The following two parameters are com‐
213 puted if JOBT .EQ. 'T'. They are provided for a devel‐
214 oper/implementer who is familiar with the details of the
215 method. WORK(6) = the entropy of A^t*A :: this is the Shannon
216 entropy of diag(A^t*A) / Trace(A^t*A) taken as point in the
217 probability simplex. WORK(7) = the entropy of A*A^t.
218 LWORK (input) INTEGER
220 Length of WORK to confirm proper allocation of work space.
221 LWORK depends on the job: If only SIGMA is needed (
222 JOBU.EQ.'N', JOBV.EQ.'N' ) and
223 -> .. no scaled condition estimate required ( JOBE.EQ.'N'):
224 LWORK >= max(2*M+N,4*N+1,7). This is the minimal requirement.
225 For optimal performance (blocked code) the optimal value is
226 LWORK >= max(2*M+N,3*N+(N+1)*NB,7). Here NB is the optimal
227 block size for xGEQP3/xGEQRF. -> .. an estimate of the scaled
228 condition number of A is required (JOBA='E', 'G'). In this
229 case, LWORK is the maximum of the above and N*N+4*N, i.e. LWORK
230 >= max(2*M+N,N*N+4N,7). If SIGMA and the right singular vec‐
231 tors are needed (JOBV.EQ.'V'), -> the minimal requirement is
232 LWORK >= max(2*N+M,7). -> For optimal performance, LWORK >=
233 max(2*N+M,2*N+N*NB,7), where NB is the optimal block size. If
234 SIGMA and the left singular vectors are needed -> the minimal
235 requirement is LWORK >= max(2*N+M,7). -> For optimal perfor‐
236 mance, LWORK >= max(2*N+M,2*N+N*NB,7), where NB is the optimal
237 block size. If full SVD is needed ( JOBU.EQ.'U' or 'F',
238 JOBV.EQ.'V' ) and -> .. the singular vectors are computed with‐
239 out explicit accumulation of the Jacobi rotations, LWORK >=
240 6*N+2*N*N -> .. in the iterative part, the Jacobi rotations are
241 explicitly accumulated (option, see the description of JOBV),
242 then the minimal requirement is LWORK >= max(M+3*N+N*N,7). For
243 better performance, if NB is the optimal block size, LWORK >=
244 max(3*N+N*N+M,3*N+N*N+N*NB,7).
245 IWORK (workspace/output) INTEGER array, dimension M+3*N.
247 On exit, IWORK(1) = the numerical rank determined after the
248 initial QR factorization with pivoting. See the descriptions of
249 JOBA and JOBR. IWORK(2) = the number of the computed nonzero
250 singular values IWORK(3) = if nonzero, a warning message: If
251 IWORK(3).EQ.1 then some of the column norms of A were denormal‐
252 ized floats. The requested high accuracy is not warranted by
253 the data.
254 INFO (output) INTEGER
256 < 0 : if INFO = -i, then the i-th argument had an illegal
257 value.
258 = 0 : successfull exit;
259 > 0 : SGEJSV did not converge in the maximal allowed number
260 of sweeps. The computed values may be inaccurate.
261
263 SGEJSV implements a preconditioned Jacobi SVD algorithm. It uses
264 SGEQP3, SGEQRF, and SGELQF as preprocessors and preconditioners.
265 Optionally, an additional row pivoting can be used as a preprocessor,
266 which in some cases results in much higher accuracy. An example is
267 matrix A with the structure A = D1 * C * D2, where D1, D2 are arbitrar‐
268 ily ill-conditioned diagonal matrices and C is well-conditioned matrix.
269 In that case, complete pivoting in the first QR factorizations provides
270 accuracy dependent on the condition number of C, and independent of D1,
271 D2. Such higher accuracy is not completely understood theoretically,
272 but it works well in practice. Further, if A can be written as A =
273 B*D, with well-conditioned B and some diagonal D, then the high accu‐
274 racy is guaranteed, both theoretically and in software, independent of
275 D. For more details see [1], [2].
276 The computational range for the singular values can be the full
277 range ( UNDERFLOW,OVERFLOW ), provided that the machine arithmetic and
278 the BLAS & LAPACK routines called by SGEJSV are implemented to work in
279 that range. If that is not the case, then the restriction for safe
280 computation with the singular values in the range of normalized IEEE
281 numbers is that the spectral condition number
282 kappa(A)=sigma_max(A)/sigma_min(A) does not overflow. This code (SGE‐
283 JSV) is best used in this restricted range, meaning that singular val‐
284 ues of magnitude below ||A||_2 / SLAMCH('O') are returned as zeros. See
285 JOBR for details on this.
286 Further, this implementation is somewhat slower than the one
287 described in [1,2] due to replacement of some non-LAPACK components,
288 and because the choice of some tuning parameters in the iterative part
289 (SGESVJ) is left to the implementer on a particular machine.
290 The rank revealing QR factorization (in this code: SGEQP3) should be
291 implemented as in [3]. We have a new version of SGEQP3 under develop‐
292 ment that is more robust than the current one in LAPACK, with a cleaner
293 cut in rank defficient cases. It will be available in the SIGMA library
294 [4]. If M is much larger than N, it is obvious that the inital QRF
295 with column pivoting can be preprocessed by the QRF without pivoting.
296 That well known trick is not used in SGEJSV because in some cases heavy
297 row weighting can be treated with complete pivoting. The overhead in
298 cases M much larger than N is then only due to pivoting, but the bene‐
299 fits in terms of accuracy have prevailed. The implementer/user can
300 incorporate this extra QRF step easily. The implementer can also
301 improve data movement (matrix transpose, matrix copy, matrix transposed
302 copy) - this implementation of SGEJSV uses only the simplest, naive
303 data movement. Contributors
304 Zlatko Drmac (Zagreb, Croatia) and Kresimir Veselic (Hagen, Germany)
305 References
306 SIAM J. Matrix Anal. Appl. Vol. 35, No. 2 (2008), pp. 1322-1342.
307 LAPACK Working note 169.
308 SIAM J. Matrix Anal. Appl. Vol. 35, No. 2 (2008), pp. 1343-1362.
309 LAPACK Working note 170.
310 factorization software - a case study.
311 ACM Trans. math. Softw. Vol. 35, No 2 (2008), pp. 1-28.
312 LAPACK Working note 176.
313 QSVD, (H,K)-SVD computations.
314 Department of Mathematics, University of Zagreb, 2008. Bugs, exam‐
315 ples and comments
316 Please report all bugs and send interesting examples and/or comments to
317 drmac@math.hr. Thank you.
318 LAPACK routine (version 3.2) November 2008 SGEJSV(1)