Properties of the A-transpose-A matrix
I believe that $A^TA$ is a key matrix structure because of its connection to variance-covariance matrices.
In Professor Strang's linear algebra lectures, "A-transpose-A" - with this nomenclature, as opposed to $X'X$, for example - is the revolving axis.
Yet, it is not easy to find on a quick Google search a list of its properties. I presume that part of the reason may be that they are shared by variance-covariance matrices. But I'd like to confirm this (does it have identical properties to a var-cov matrix?), and have the list easily available from now on here at SE-Mathematics.
Just to not shy away from the initial effort, here is what I think I have so far:
- Symmetry
- Positive semidefinite-ness
- Real and positive eigenvalues
- The trace is positive (the trace is the sum of eigenvalues)
- The determinant is positive (the determinant is the product of the eigenvalues)
- The diagonal entries are all positive
- Orthogonal eigenvectors (**)
- Diagonalizable as $Q\Lambda Q^T$
- It is possible to obtain a Cholesky decomposition.
- Rank of $A^TA$ is the same as rank of $A$.
- $\text{ker}(A^TA)=\text{ker}(A)$
(**) The eigenvectors of A-transpose-A form the matrix $V$ in singular value decomposition (SVD) of $A,$ while the square root of the eigenvalues of A-transpose-A are the singular values of the SVD. Similarly, the eigenvectors of A-A-transpose $AA^\top$ include the columns in the matrix $U$ of the SVD of $A.$ The importance of this is exemplified in the fact that SVD can be used to solve least squares regression by computing the Penrose-Moore pseudo-inverse $A^\dagger = V\Sigma^\dagger U^*,$ although the QR decomposition is a more expedient computational method.
There is a nice post on the topic here.
$\endgroup$ 51 Answer
$\begingroup$Yes, it has all the properties of a covariance matrix because it is one. You can define a multivariate normal distribution for which $A^T A$ is the covariance matrix.
$\endgroup$ 2
