A note on stable perturbations of Moore-Penrose ... - Semantic Scholar
NUMERICAL LINEAR ALGEBRA WITH APPLICATIONS Numer. Linear Algebra Appl. 2011; 00:1–8
Prepared using nlaauth.cls [Version: 2000/03/22 v1.0]
A note on stable perturbations of Moore-Penrose inverses† Zhao Li, 1
1
Qingxiang Xu,
2,3
and Yimin Wei
1,∗
School of Mathematical Sciences & Shanghai Key Laboratory of Contemporary Applied Mathematics, Fudan University, Shanghai, 200433, People’s Republic of China. Department of Mathematics, Shanghai Normal University, Shanghai 200234, People’s Republic of China. 3 College of Sciences, Shanghai Institute of Technology, Shanghai 200235, People’s Republic of China.
2
SUMMARY Perturbation bounds for Moore-Penrose inverses of rectangular matrices play a significant role in the perturbation analysis for linear least squares problems. In this note, we derive a sharp upper bound for Moore-Penrose inverses, c 2011 John Wiley & Sons, Ltd. which is better than a well known existing one [12]. Copyright
Keywords: Moore-Penrose inverse, acute perturbation, stable perturbation.
1. Introduction Throughout this note, k · k means the spectral norm. For a matrix A ∈ Rm×n , the Moore-Penrose inverse A† of A is defined as the unique n × m matrix satisfying the following four matrix equations ([1, 2, 3]), AA† A = A,
A† AA† = A† ,
(AA† )T = AA† ,
(A† A)T = A† A.
(1.1)
¯ = A + ∆A ∈ Rm×n be a matrix perturbed from A. Pursuing an upper bound for kA¯† k plays a significant Let A role in investigating the perturbation behavior of linear least squares problems ([1, 2, 4, 5, 6, 7, 8, 9, 10]). Recalling the fact that an arbitrarily small perturbation of A can change the rank of A and cause an arbitrarily large perturbation of the Moore-Penrose inverse, there is no wonder that all the known bounds have been established under specific constraints of the way A is perturbed. Stewart ([11, 12]) and Wedin ([13]) independently investigated the constraint of the so called acute perturbation.
∗ Correspondence
to: Yimin Wei, School of Mathematical Sciences, Fudan University, Shanghai, 200433, People’s Republic
of China. Contract/grant sponsor: Z. Li is supported by Doctoral Program of the Ministry of Education under grant 20090071110003 and 973 Program Project under grant 2010CB327900. Contract/grant sponsor: Q. Xu is supported by the the National Natural Science Foundation of China under grant 11171222, and the Innovation Program of Shanghai Municipal Education Commission under grant 12ZZ129. Contract/grant sponsor: Y. Wei is supported by the National Natural Science Foundation of China under grant 10871051, Doctoral Program of the Ministry of Education under grant 20090071110003, Shanghai Education Committee and Shanghai Science & Technology Committee. † E-mail: [email protected]; [email protected]; [email protected] and [email protected]
c 2011 John Wiley & Sons, Ltd. Copyright
Received March, 2009 Revised May and October, 2010; June and November, 2011
2
Z. LI AND Q. XU AND AND Y. WEI
Definition 1. ([12, 13]) A matrix A¯ = A + ∆A ∈ Rm×n is said to be an acute perturbation of A ∈ Rm×n , if
kPA − PA¯ k < 1 and kPAT − PA¯T k < 1, where PA = AA† and PAT = A† A stand for the two orthogonal projectors onto R(A) (the range space of A), and R(AT ), respectively. Chen and Xue et al. introduced the concept of the stable perturbation.
Definition 2. ([14, 15]) Let A¯ = A + ∆A ∈ Rm×n be a perturbation of A. Then A¯ is said to be a stable ¯ ∩ R⊥ (A) = {0}, R⊥ (A) denotes the orthogonal complementary subspace of R(A). perturbation of A if R(A)
¯ But A is not necessarily an acute If A¯ is an acute perturbation of A, then A is an stable perturbation of A. ¯ when A¯ is a stable perturbation of A. Fortunately, it has been proved that if k∆Ak is small perturbation of A, enough, then stable perturbation implies acute perturbation. The result is presented as the following lemma.
Lemma 1. ([16, Theorem 1], [17, Lemma 1 and Lemma 3 and Theorem 4]) Let A¯ = A + ∆A ∈ Rm×n be a perturbation of A ∈ Rm×n . If kA† k k∆Ak < 1, then the following statements are equivalent: (i) A¯ is a stable perturbation of A;
¯ is an acute perturbation of A; (ii) A
¯ (iii) rank(A) = rank(A).
¯ is an acute perturbation of A and kA† k k∆Ak < 1, then kA ¯† k has the well known estimation Note that if A ([12, 13]), kA¯† k ≤ We obtain a sharp upper bound for kA¯† k as follows, kA¯† k ≤ µ Furthermore, µ 2 , µ can be arbitrarily small in the following example. 1 0 ¯ is a stable perturbation of A and ¯ = ta aa , where 0 < a, t < 1 and a = t − t2 , then A Let A = and A 0 0 a t † 1 µ = 1+(1/t) 2 . The condition kA k k∆Ak < 1 holds if and only if k∆Ak = σmax (∆A) < 1, where σmax (∆A) denotes the largest singular value of the matrix ∆A. σmax (∆A) is the root with the larger absolute value of the two roots of the quadratic equation det (λI − ∆A) = 0, which can be rewritten explicitly as λ2 − (ta + at − 1)λ − at = 0. Since p ta + at − 1 = −t + t2 − t3 < 0, we have σmax (∆A) = − 12 ta + at − 1 − (ta + at − 1)2 + 4 at . Then σmax (∆A) < 1 p if and only if ta + at + 1 > (ta + at − 1)2 + 4 at , which can be simplified as t · a > 0. Recall that t, a > 0, the condition kA† k k∆Ak < 1 is always satisfied. We can see that µ → 0 while t → 0.
4. Application Now we will present the application of our results to the perturbation for the least squares solution x = A† b, kAx − bk = minn kAv − bk.
(4.1)
v∈R
¯† (b + ∆b) be the least squares solution of perturbed one Suppose that x ¯ = A¯†¯b = A ¯ − ¯bk. ¯ − ¯bk = min kAv kAx n
(4.2)
v∈R
We need a classical lemma which is due to Wedin in [13, Theorem 5.1].
Lemma 6. If A¯ = A + ∆A is an acute perturbation of A, satisfying kA† k k∆Ak < 1, x, x¯ are the least squares solutions of problems (4.1) and (4.2) respectively, then
†T ¯† k (k∆Ak kxk + k∆bk + krk) + k∆Ak k¯ x − xk ≤ kA
A x
kA† k
≤ (k∆Ak kxk + k∆bk + krk) + k∆Ak A†T x (4.3) † 1 − kA k k∆Ak where r = b − Ax.
¯† k in Theorem 2 into Eqn. (4.3), we obtain the following theorem. By substituting the estimation of kA
Theorem 3. If A¯ = A + ∆A is an acute perturbation of A, satisfying kA† k k∆Ak < 1, x, x¯ are the solutions of the least squares problems (4.1) and (4.2) respectively, then k¯ x − xk ≤ µ
kA† k
†T (k∆Ak kxk + k∆bk + krk) + k∆Ak x
A
1 − kA† k k∆Ak
(4.4)
where r = b − Ax and µ = k(I2r + Z1 Z1T )−1 k ≤ 1.
Remark 2. In this note we present an upper bound for Moore-Penrose inverses of rectangular matrices, which is sharper than a well known existing one [12], and can be extended to bounded linear operators on Hilbert spaces. c 2011 John Wiley & Sons, Ltd. Copyright Prepared using nlaauth.cls
Numer. Linear Algebra Appl. 2011; 00:1–8
8
Z. LI AND Q. XU AND AND Y. WEI
Acknowledgements. We are grateful for Prof. Owe Axelsson, Dr. Maya Neytcheva and two referees for their valuable comments and useful suggestions which greatly improved the presentation of our paper. One referee provided us with another proof of Theorem 1.
REFERENCES ˚. Bj¨ 1. A orck, Numerical Methods for Least Squares Problems, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1996. 2. C. L. Lawson and R. J. Hanson, Solving Least Squares Problems, Revised reprint of the 1974 original; Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1995. 3. G. Wang, Y. Wei and S. Qiao, Generalized Inverses: Theory and Computations, Science Press, Bejing, 2004. 4. M. Arioli, M. Baboulin and S. Gratton, A partial condition number for linear least-squares problems, SIAM Journal on Matrix Analysis and Applications, 29 (2007), pp. 413-433. 5. M. Baboulin, J. Dongarra, S. Gratton and J. Langou, Computing the conditioning of the components of a linear least-squares solution, Numerical Linear Algebra with Applications, 16 (2009), pp. 517-533. 6. M. Baboulin and S. Gratton, Using dual techniques to derive componentwise and mixed condition numbers for a linear function of a linear least squares solution, BIT, 49 (2009), pp. 3-19. 7. F. Cucker, H. Diao and Y. Wei, On mixed and componentwise condition numbers for Moore-Penrose inverse and linear least squares problems, Mathematics of Computation, 76 (2007), 947-963. 8. J. Demmel, Y. Hida, X. S. Li and E. Riedy, Extra-precise iterative refinement for overdetermined least squares problems, ACM Transactions on Mathematical Software, 35 (2009), Article No. 28. 9. M. Gulliksson, X. Jin and Y. Wei, Perturbation bounds for constrained and weighted least squares problems, Linear Algebra and Its Applications, 349 (2002), 221-232. 10. H. Diao and Y. Wei, On Frobenius normwise condition numbers for Moore-Penrose inverse and linear least-squares problems, Numerical Linear Algebra with Applications, 14 (2007), 603-610. 11. G. W. Stewart, On the continuity of the the generalized inverse, SIAM Journal on Applied Mathematics, 17 (1969), 33-45. 12. G. W. Stewart, On the perturbation of pseudo-inverse,projections and linear least squares problems, SIAM Review, 19 (1977), 634-662. 13. P. ˚ A. Wedin, Perturbation theory for pseudo-inverses, BIT, 13 (1973), 217-232. 14. G. Chen, M. Wei and Y. Xue, Perturbation analysis of the least squares solution in Hilbert spaces, Linear Algebra and Its Applications, 244 (1996), 69-80. 15. G. Chen and Y. Xue, Perturbation analysis for the operator equation T x = b in Banach spaces, Journal of Mathematical Analysis and Applications, 212 (1997), 107-125. 16. G. Chen and Y. Xue, The expression of the generalized inverse of the perturbed operator under Type I perturbation in Hilbert spaces, Linear Algebra and Its Applications, 285 (1998), 1-6. 17. Y. Xue and G. Chen, Some equivalent conditions of stable perturbation of operators in Hilbert spaces, Applied Mathematics and Computation, 147 (2004), 765-772. 18. G. W. Stewart and J.-G. Sun, Matrix Perturbation Theory, Academic Press, New York, 1990.
c 2011 John Wiley & Sons, Ltd. Copyright Prepared using nlaauth.cls
Numer. Linear Algebra Appl. 2011; 00:1–8
Prepared using nlaauth.cls [Version: 2000/03/22 v1.0]
A note on stable perturbations of Moore-Penrose inverses† Zhao Li, 1
1
Qingxiang Xu,
2,3
and Yimin Wei
1,∗
School of Mathematical Sciences & Shanghai Key Laboratory of Contemporary Applied Mathematics, Fudan University, Shanghai, 200433, People’s Republic of China. Department of Mathematics, Shanghai Normal University, Shanghai 200234, People’s Republic of China. 3 College of Sciences, Shanghai Institute of Technology, Shanghai 200235, People’s Republic of China.
2
SUMMARY Perturbation bounds for Moore-Penrose inverses of rectangular matrices play a significant role in the perturbation analysis for linear least squares problems. In this note, we derive a sharp upper bound for Moore-Penrose inverses, c 2011 John Wiley & Sons, Ltd. which is better than a well known existing one [12]. Copyright
Keywords: Moore-Penrose inverse, acute perturbation, stable perturbation.
1. Introduction Throughout this note, k · k means the spectral norm. For a matrix A ∈ Rm×n , the Moore-Penrose inverse A† of A is defined as the unique n × m matrix satisfying the following four matrix equations ([1, 2, 3]), AA† A = A,
A† AA† = A† ,
(AA† )T = AA† ,
(A† A)T = A† A.
(1.1)
¯ = A + ∆A ∈ Rm×n be a matrix perturbed from A. Pursuing an upper bound for kA¯† k plays a significant Let A role in investigating the perturbation behavior of linear least squares problems ([1, 2, 4, 5, 6, 7, 8, 9, 10]). Recalling the fact that an arbitrarily small perturbation of A can change the rank of A and cause an arbitrarily large perturbation of the Moore-Penrose inverse, there is no wonder that all the known bounds have been established under specific constraints of the way A is perturbed. Stewart ([11, 12]) and Wedin ([13]) independently investigated the constraint of the so called acute perturbation.
∗ Correspondence
to: Yimin Wei, School of Mathematical Sciences, Fudan University, Shanghai, 200433, People’s Republic
of China. Contract/grant sponsor: Z. Li is supported by Doctoral Program of the Ministry of Education under grant 20090071110003 and 973 Program Project under grant 2010CB327900. Contract/grant sponsor: Q. Xu is supported by the the National Natural Science Foundation of China under grant 11171222, and the Innovation Program of Shanghai Municipal Education Commission under grant 12ZZ129. Contract/grant sponsor: Y. Wei is supported by the National Natural Science Foundation of China under grant 10871051, Doctoral Program of the Ministry of Education under grant 20090071110003, Shanghai Education Committee and Shanghai Science & Technology Committee. † E-mail: [email protected]; [email protected]; [email protected] and [email protected]
c 2011 John Wiley & Sons, Ltd. Copyright
Received March, 2009 Revised May and October, 2010; June and November, 2011
2
Z. LI AND Q. XU AND AND Y. WEI
Definition 1. ([12, 13]) A matrix A¯ = A + ∆A ∈ Rm×n is said to be an acute perturbation of A ∈ Rm×n , if
kPA − PA¯ k < 1 and kPAT − PA¯T k < 1, where PA = AA† and PAT = A† A stand for the two orthogonal projectors onto R(A) (the range space of A), and R(AT ), respectively. Chen and Xue et al. introduced the concept of the stable perturbation.
Definition 2. ([14, 15]) Let A¯ = A + ∆A ∈ Rm×n be a perturbation of A. Then A¯ is said to be a stable ¯ ∩ R⊥ (A) = {0}, R⊥ (A) denotes the orthogonal complementary subspace of R(A). perturbation of A if R(A)
¯ But A is not necessarily an acute If A¯ is an acute perturbation of A, then A is an stable perturbation of A. ¯ when A¯ is a stable perturbation of A. Fortunately, it has been proved that if k∆Ak is small perturbation of A, enough, then stable perturbation implies acute perturbation. The result is presented as the following lemma.
Lemma 1. ([16, Theorem 1], [17, Lemma 1 and Lemma 3 and Theorem 4]) Let A¯ = A + ∆A ∈ Rm×n be a perturbation of A ∈ Rm×n . If kA† k k∆Ak < 1, then the following statements are equivalent: (i) A¯ is a stable perturbation of A;
¯ is an acute perturbation of A; (ii) A
¯ (iii) rank(A) = rank(A).
¯ is an acute perturbation of A and kA† k k∆Ak < 1, then kA ¯† k has the well known estimation Note that if A ([12, 13]), kA¯† k ≤ We obtain a sharp upper bound for kA¯† k as follows, kA¯† k ≤ µ Furthermore, µ 2 , µ can be arbitrarily small in the following example. 1 0 ¯ is a stable perturbation of A and ¯ = ta aa , where 0 < a, t < 1 and a = t − t2 , then A Let A = and A 0 0 a t † 1 µ = 1+(1/t) 2 . The condition kA k k∆Ak < 1 holds if and only if k∆Ak = σmax (∆A) < 1, where σmax (∆A) denotes the largest singular value of the matrix ∆A. σmax (∆A) is the root with the larger absolute value of the two roots of the quadratic equation det (λI − ∆A) = 0, which can be rewritten explicitly as λ2 − (ta + at − 1)λ − at = 0. Since p ta + at − 1 = −t + t2 − t3 < 0, we have σmax (∆A) = − 12 ta + at − 1 − (ta + at − 1)2 + 4 at . Then σmax (∆A) < 1 p if and only if ta + at + 1 > (ta + at − 1)2 + 4 at , which can be simplified as t · a > 0. Recall that t, a > 0, the condition kA† k k∆Ak < 1 is always satisfied. We can see that µ → 0 while t → 0.
4. Application Now we will present the application of our results to the perturbation for the least squares solution x = A† b, kAx − bk = minn kAv − bk.
(4.1)
v∈R
¯† (b + ∆b) be the least squares solution of perturbed one Suppose that x ¯ = A¯†¯b = A ¯ − ¯bk. ¯ − ¯bk = min kAv kAx n
(4.2)
v∈R
We need a classical lemma which is due to Wedin in [13, Theorem 5.1].
Lemma 6. If A¯ = A + ∆A is an acute perturbation of A, satisfying kA† k k∆Ak < 1, x, x¯ are the least squares solutions of problems (4.1) and (4.2) respectively, then
†T ¯† k (k∆Ak kxk + k∆bk + krk) + k∆Ak k¯ x − xk ≤ kA
A x
kA† k
≤ (k∆Ak kxk + k∆bk + krk) + k∆Ak A†T x (4.3) † 1 − kA k k∆Ak where r = b − Ax.
¯† k in Theorem 2 into Eqn. (4.3), we obtain the following theorem. By substituting the estimation of kA
Theorem 3. If A¯ = A + ∆A is an acute perturbation of A, satisfying kA† k k∆Ak < 1, x, x¯ are the solutions of the least squares problems (4.1) and (4.2) respectively, then k¯ x − xk ≤ µ
kA† k
†T (k∆Ak kxk + k∆bk + krk) + k∆Ak x
A
1 − kA† k k∆Ak
(4.4)
where r = b − Ax and µ = k(I2r + Z1 Z1T )−1 k ≤ 1.
Remark 2. In this note we present an upper bound for Moore-Penrose inverses of rectangular matrices, which is sharper than a well known existing one [12], and can be extended to bounded linear operators on Hilbert spaces. c 2011 John Wiley & Sons, Ltd. Copyright Prepared using nlaauth.cls
Numer. Linear Algebra Appl. 2011; 00:1–8
8
Z. LI AND Q. XU AND AND Y. WEI
Acknowledgements. We are grateful for Prof. Owe Axelsson, Dr. Maya Neytcheva and two referees for their valuable comments and useful suggestions which greatly improved the presentation of our paper. One referee provided us with another proof of Theorem 1.
REFERENCES ˚. Bj¨ 1. A orck, Numerical Methods for Least Squares Problems, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1996. 2. C. L. Lawson and R. J. Hanson, Solving Least Squares Problems, Revised reprint of the 1974 original; Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1995. 3. G. Wang, Y. Wei and S. Qiao, Generalized Inverses: Theory and Computations, Science Press, Bejing, 2004. 4. M. Arioli, M. Baboulin and S. Gratton, A partial condition number for linear least-squares problems, SIAM Journal on Matrix Analysis and Applications, 29 (2007), pp. 413-433. 5. M. Baboulin, J. Dongarra, S. Gratton and J. Langou, Computing the conditioning of the components of a linear least-squares solution, Numerical Linear Algebra with Applications, 16 (2009), pp. 517-533. 6. M. Baboulin and S. Gratton, Using dual techniques to derive componentwise and mixed condition numbers for a linear function of a linear least squares solution, BIT, 49 (2009), pp. 3-19. 7. F. Cucker, H. Diao and Y. Wei, On mixed and componentwise condition numbers for Moore-Penrose inverse and linear least squares problems, Mathematics of Computation, 76 (2007), 947-963. 8. J. Demmel, Y. Hida, X. S. Li and E. Riedy, Extra-precise iterative refinement for overdetermined least squares problems, ACM Transactions on Mathematical Software, 35 (2009), Article No. 28. 9. M. Gulliksson, X. Jin and Y. Wei, Perturbation bounds for constrained and weighted least squares problems, Linear Algebra and Its Applications, 349 (2002), 221-232. 10. H. Diao and Y. Wei, On Frobenius normwise condition numbers for Moore-Penrose inverse and linear least-squares problems, Numerical Linear Algebra with Applications, 14 (2007), 603-610. 11. G. W. Stewart, On the continuity of the the generalized inverse, SIAM Journal on Applied Mathematics, 17 (1969), 33-45. 12. G. W. Stewart, On the perturbation of pseudo-inverse,projections and linear least squares problems, SIAM Review, 19 (1977), 634-662. 13. P. ˚ A. Wedin, Perturbation theory for pseudo-inverses, BIT, 13 (1973), 217-232. 14. G. Chen, M. Wei and Y. Xue, Perturbation analysis of the least squares solution in Hilbert spaces, Linear Algebra and Its Applications, 244 (1996), 69-80. 15. G. Chen and Y. Xue, Perturbation analysis for the operator equation T x = b in Banach spaces, Journal of Mathematical Analysis and Applications, 212 (1997), 107-125. 16. G. Chen and Y. Xue, The expression of the generalized inverse of the perturbed operator under Type I perturbation in Hilbert spaces, Linear Algebra and Its Applications, 285 (1998), 1-6. 17. Y. Xue and G. Chen, Some equivalent conditions of stable perturbation of operators in Hilbert spaces, Applied Mathematics and Computation, 147 (2004), 765-772. 18. G. W. Stewart and J.-G. Sun, Matrix Perturbation Theory, Academic Press, New York, 1990.
c 2011 John Wiley & Sons, Ltd. Copyright Prepared using nlaauth.cls
Numer. Linear Algebra Appl. 2011; 00:1–8
