Similarity transformation

A similarity transformation of a matrix $A$ to a matrix $A'$ is expressed as $A'=P^{-1}AP$ , where

1. $P$ is an invertible matrix called the change of basis matrix.
2. $A$ is a linear transformation matrix with respect to the basis $\boldsymbol{\mathit{e}}$ .
3. $A'$ is the transformed representation of $A$ , such that $A'$ performs the same linear transformation as $A$ but with respect to another basis $\boldsymbol{\mathit{e}}'$ .

Let a vector be $\boldsymbol{\mathit{x}}$ with respect to the basis $\boldsymbol{\mathit{e}}$ and $\boldsymbol{\mathit{x}}'$ with respect to the basis $\boldsymbol{\mathit{e}}'$ . Let another vector be $\boldsymbol{\mathit{y}}$ with respect to the basis $\boldsymbol{\mathit{e}}$ and $\boldsymbol{\mathit{y}}'$ with respect to the basis $\boldsymbol{\mathit{e}}'$ . Consider the transformation of these vectors as follows:

$\boldsymbol{\mathit{x}}=P\boldsymbol{\mathit{x}}'\;\;\;\;\;\;\boldsymbol{\mathit{y}}=P\boldsymbol{\mathit{y}}'\;\;\;\;\;\;\boldsymbol{\mathit{x}}=A\boldsymbol{\mathit{y}}$

where the first two equations describe change of basis transformations and the last equation is a linear transformation of $\boldsymbol{\mathit{y}}$ to $\boldsymbol{\mathit{x}}$ in the same basis $\boldsymbol{\mathit{e}}$ .

Combining the three equation, we have $P\boldsymbol{\mathit{x}}'=A\boldsymbol{\mathit{y}}=AP\boldsymbol{\mathit{y}}'$ . If $P$ is invertible, we can multiply $P\boldsymbol{\mathit{x}}'=AP\boldsymbol{\mathit{y}}'$ by $P^{-1}$ on the left to give $\boldsymbol{\mathit{x}}'=A'\boldsymbol{\mathit{y}}'$ , where $A'=P^{-1}AP$ . Comparing $\boldsymbol{\mathit{x}}=A\boldsymbol{\mathit{y}}$ and $\boldsymbol{\mathit{x}}'=A'\boldsymbol{\mathit{y}}'$ , $A'$ is the transformed representation of $A$ , where $A'$ performs the same linear transformation as $A$ but with respect to another basis $\boldsymbol{\mathit{e}}'$ . We say that $A'$ is similar to $A$ because $A'$ has properties that are similar to $A$ . For example, the trace of $A'$ , which is defined as $Tr(A')=\sum_ia_{ii}'$ , is the same as the trace of $A$ .

To show that $Tr(A')=Tr(A)$ , we have,

$Tr(A')=Tr(G^{-1}AG)=Tr(GG^{-1}A)=Tr(EA)=Tr(A)$

where $E$ is the identity matrix and where we have used the identity $Tr(XY)=Tr(YX)$ for the second equality.

Question

Proof that $Tr(XY)=Tr(YX)$ .

Answer

$Tr(XY)=(XY)_{11}+(XY)_{22}+\cdots +(XY)_{nn}$

$=\begin{matrix}x_{11}y_{11}+&x_{12}y_{21}+&\cdots&x_{1k}y_{k1}\\+x_{21}y_{12}+&x_{22}y_{22}+&\cdots&x_{2k}y_{k2}\\\vdots&\vdots&\vdots&\vdots\\+x_{n1}y_{1n}+&x_{n2}y_{2n}+&\cdots&x_{nk}y_{kn} \end{matrix}=\begin{matrix}x_{11}y_{11}+&x_{21}y_{12}+&\cdots&x_{n1}y_{1n}\\+x_{12}y_{21}+&x_{22}y_{22}+&\cdots&x_{n2}y_{2n}\\\vdots&\vdots&\vdots&\vdots\\+x_{1k}y_{k1}+&x_{2k}y_{k2}+&\cdots&x_{nk}y_{kn} \end{matrix}$

$=\begin{matrix}y_{11}x_{11}+&y_{12}x_{21}+&\cdots&y_{1n}x_{n1}\\+y_{21}x_{12}+&y_{22}x_{22}+&\cdots&y_{2n}x_{n2}\\\vdots&\vdots&\vdots&\vdots\\+y_{k1}x_{1k}+&y_{k2}x_{2k}+&\cdots&y_{kn}x_{nk} \end{matrix}=(YX)_{11}+(YX)_{22}+\cdots+(YX)_{nn}\\=Tr(YX)$

The common properties of similar matrices are useful for explaining certain group theory concepts, such as why there are exactly 32 crystallographic point groups.

One of the most common applications of similarity transformations is to transform a matrix to a diagonal matrix $D$ . Consider the eigenvalue problem $A\boldsymbol{\mathit{u}}_j=\lambda_j\boldsymbol{\mathit{u}}_j$ and let the eigenvectors of $A$ be the columns of $P$ :

$P=(\boldsymbol{\mathit{u}}_1,\boldsymbol{\mathit{u}}_2,\cdots ,\boldsymbol{\mathit{u}}_n)=\begin{pmatrix}u_{11} & u_{21} & \cdots & u_{n1} \\u_{12} & u_{22} & \cdots & u_{n2}\\ \vdots & \vdots & \ddots & \vdots \\u_{1n} & u_{2n} & \cdots & u_{nn} \end{pmatrix}$

Since $A=\begin{pmatrix}u_{j1}\\ u_{j2}\\ \vdots\\ u_{jn}\end{pmatrix}=\lambda_j\begin{pmatrix}u_{j1}\\ u_{j2}\\ \vdots\\ u_{jn} \end{pmatrix}$ , where $A=\begin{pmatrix}a_{11} & a_{12} & \cdots & a_{1n} \\a_{21} & a_{22} & \cdots & a_{2n}\\ \vdots & \vdots & \ddots & \vdots \\a_{n1} & a_{n2} & \cdots & a_{nn} \end{pmatrix}$ , we have

$AP=\begin{pmatrix}\lambda_1u_{11} &\lambda_2u_{21} & \cdots &\lambda_nu_{n1} \\\lambda_1u_{12} &\lambda_2u_{22} & \cdots &\lambda_nu_{n2}\\ \vdots & \vdots & \ddots & \vdots \\\lambda_1u_{1n} &\lambda_2u_{2n} & \cdots &\lambda_nu_{nn} \end{pmatrix}=P\begin{pmatrix}\lambda_1 &0 & \cdots &0 \\0 &\lambda_2 & \cdots &0\\ \vdots & \vdots & \ddots & \vdots \\\0 &0 & \cdots &\lambda_n \end{pmatrix}=PD$

If the eigenvectors of $A$ are linearly independent, then $P$ is non-singular (i.e. invertible). This allows us to multiply $AP=PD$ on the left by $P^{-1}$ to give $D=P^{-1}AP$ , with the diagonal entries of $D$ being the eigenvalues of $A$ .

Question

Why is $P$ non-singular if the eigenvectors of $A$ are linearly independent?

Answer

The eigenvectors $\boldsymbol{\mathit{u}}_1,\boldsymbol{\mathit{u}}_2,\cdots,\boldsymbol{\mathit{u}}_n$ are linearly independent if the only solution to $x_1\boldsymbol{\mathit{u}}_1+x_2\boldsymbol{\mathit{u}}_2+\cdots+x_n\boldsymbol{\mathit{u}}_n=\boldsymbol{\mathit{0}}$ is when $x_n=0$ for all $n$ . In other words,

$(\boldsymbol{\mathit{u}}_1,\boldsymbol{\mathit{u}}_2,\cdots,\boldsymbol{\mathit{u}}_n)\begin{pmatrix}x_1 \\x_2 \\\vdots \\ x_n \end{pmatrix}=\boldsymbol{\mathit{0}}$

$\begin{pmatrix}u_{11} & u_{21} & \cdots & u_{n1} \\u_{12} & u_{22} & \cdots & u_{n2}\\ \vdots & \vdots & \ddots & \vdots \\u_{1n} & u_{2n} & \cdots & u_{nn} \end{pmatrix}\begin{pmatrix}x_1 \\x_2 \\\vdots \\ x_n \end{pmatrix}=\begin{pmatrix}0 \\0 \\\vdots \\ 0 \end{pmatrix}$

or simply $Px=0$ .

We need to show that the only solution to $Px=0$ is $x=0$ and this is possible if $P$ is invertible such that

$Px=0\implies P^{-1}Px=P^{-1}0\implies x=0$

Using the same eigenvalue problem, we can show that Hermitian operators $\hat{H}$ are diagonalisable, i.e. $D=P^{-1}\hat{H}P$ (see this article).

Question

Show that a Hermitian matrix $H$ can be diagonalised by $U$ , i.e. $D=U^{-1}HU$ , where $U$ is a unitary matrix, and that $D$ is also Hermitian.

Answer

A unitary matrix has the property: $U^{\dagger}U=UU^{\dagger}=I$ . If a complete set of orthonormal eigenvectors of $H$ are the columns of $P$ , we have $HP=PD$ and $D=P^{-1}HP$ because orthonormal eigenvectors are linearly independent. The remaining step is to show that $P^{\dagger}P=PP^{\dagger}=I$ .

$P^{\dagger}=\begin{pmatrix}p_{11} & p_{21} & \cdots & p_{n1} \\p_{12} & p_{22} & \cdots & p_{n2}\\ \vdots & \vdots & \ddots & \vdots \\p_{1n} & p_{2n} & \cdots & p_{nn} \end{pmatrix}^{\dagger}=\begin{pmatrix}p_{11}^* & p_{12}^* & \cdots & p_{1n}^* \\p_{21}^* & p_{22}^* & \cdots & p_{2n}^*\\ \vdots & \vdots & \ddots & \vdots \\p_{n1}^* & p_{n2}^* & \cdots & p_{nn}^* \end{pmatrix}$

$P^{\dagger}P=\begin{pmatrix}p_{11}^* & p_{12}^* & \cdots & p_{1n}^* \\p_{21}^* & p_{22}^* & \cdots & p_{2n}^*\\ \vdots & \vdots & \ddots & \vdots \\p_{n1}^* & p_{n2}^* & \cdots & p_{nn}^* \end{pmatrix}\begin{pmatrix}p_{11} & p_{21} & \cdots & p_{n1} \\p_{12} & p_{22} & \cdots & p_{n2}\\ \vdots & \vdots & \ddots & \vdots \\p_{1n} & p_{2n} & \cdots & p_{nn} \end{pmatrix}\\=(\langle\boldsymbol{\mathit{p}}_i\vert\boldsymbol{\mathit{p}}_j\rangle)_{ij}\delta_{ij}=I$

For example, $\begin{pmatrix}0&1&0\\1&0&0\\0&0&1\end{pmatrix}\begin{pmatrix}0&1&0\\1&0&0\\0&0&1\end{pmatrix}=\begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}$ . Since $P$ is non-singular, multiplying $P^{-1}$ on the right of $P^{\dagger}P=I$ gives $P^{\dagger}=P^{-1}$ . So, $I=PP^{-1}=PP^{\dagger}$ .

To show that $D$ is also Hermitian, we have

$D^{\dagger}=(U^{\dagger}HU)^{\dagger}=U^{\dagger}H^{\dagger}U=U^{\dagger}HU=D$ .

As mentioned above, $D=P^{-1}AP\implies AP=PD$ . The $i$ -th column $c_i$ of $AP$ is $Ac_i$ , while the $i$ -th column of $PD$ is $D_{ii}c_i$ . So, $Ac_i=D_{ii}c_i$ , where $c_i$ is an eigenvector of $A$ and $D_{ii}$ is the corresponding eigenvalue. Therefore, the order of the columns of the change of basis matrix corresponds to the order of the diagonal entries in the diagonal matrix.