Gram-Schmidt Process

The Gram-Schmidt process is a mathematical technique for orthogonalising a set of nonorthogonal vectors.

Consider two linearly independent nonorthogonal vectors $\boldsymbol{\mathit{ x}}_1$ and $\boldsymbol{\mathit{ x}}_2$ of the Hermitian operator $\hat O$ :

$\hat O\boldsymbol{\mathit{ x}}_1=a\boldsymbol{\mathit{ x}}_1$

$\hat O\boldsymbol{\mathit{ x}}_2=a\boldsymbol{\mathit{ x}}_2$

where $a$ is the degenerate eigenvalue corresponding to $\boldsymbol{\mathit{ x}}_1$ and $\boldsymbol{\mathit{ x}}_2$ .

Let $\boldsymbol{\mathit{ x}}_1=\boldsymbol{\mathit{ x}}_1^{'}$ . With reference to the diagram above, $\vert\boldsymbol{\mathit{ x}}_{2p}\vert=\vert\boldsymbol{\mathit{ x}}_2\vert cos\theta=\boldsymbol{\mathit{ x}}_2\cdot\frac{\boldsymbol{\mathit{ x}}_1^{'}}{\vert\boldsymbol{\mathit{ x}}_1^{'}\vert}$ , where we have used the fact that the magnitude of the unit vector $\frac{\boldsymbol{\mathit{ x}}_1^{'}}{\vert\boldsymbol{\mathit{ x}}_1^{'}\vert}$ is 1 in the second equality. Furthermore, $\boldsymbol{\mathit{ x}}_{2p}=\vert\boldsymbol{\mathit{ x}}_{2p}\vert \frac{\boldsymbol{\mathit{ x}}_1^{'}}{\vert\boldsymbol{\mathit{ x}}_1^{'}\vert}$ ; that is, the vector $\boldsymbol{\mathit{ x}}_{2p}$ is a multiple of the unit vector $\frac{\boldsymbol{\mathit{ x}}_1^{'}}{\vert\boldsymbol{\mathit{ x}}_1^{'}\vert}$ , with the multiple being $\vert\boldsymbol{\mathit{ x}}_{2p}\vert$ . Therefore,

$\boldsymbol{\mathit{ x}}_2^{'}=\boldsymbol{\mathit{ x}}_2-\boldsymbol{\mathit{ x}}_{2p}=\boldsymbol{\mathit{ x}}_2-\vert\boldsymbol{\mathit{ x}}_{2p}\vert\frac{\boldsymbol{\mathit{ x}}_1^{'}}{\vert\boldsymbol{\mathit{ x}}_1^{'}\vert}=\boldsymbol{\mathit{ x}}_2-\left(\boldsymbol{\mathit{ x}}_2\cdot\frac{\boldsymbol{\mathit{ x}}_1^{'}}{\vert\boldsymbol{\mathit{ x}}_1^{'}\vert}\right)\frac{\boldsymbol{\mathit{ x}}_1^{'}}{\vert\boldsymbol{\mathit{ x}}_1^{'}\vert}$ $=\boldsymbol{\mathit{ x}}_2-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_2\cdot\mathit{ x}}_1^{'}}{\vert\boldsymbol{\mathit{ x}}_1^{'}\vert^2}\boldsymbol{\mathit{ x}}_1^{'}=\boldsymbol{\mathit{ x}}_2-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_2\cdot\mathit{ x}}_1^{'}}{\boldsymbol{\mathit{ x}}_1^{'}\cdot\boldsymbol{\mathit{ x}}_1^{'}}\boldsymbol{\mathit{ x}}_1^{'}\;\;\;\;\;\;\;\;(111)$
$\boldsymbol{\mathit{ x}}_2^{'}$ is a component of $\boldsymbol{\mathit{ x}}_2$ , which is orthogonal to $\boldsymbol{\mathit{ x}}_1^{'}$ . The eigenvalue of $\boldsymbol{\mathit{ x}}_2^{'}$ is unchanged versus that of $\boldsymbol{\mathit{ x}}_2$ because

$\hat O\left(\boldsymbol{\mathit{ x}}_2-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_2\cdot\mathit{ x}}_1^{'}}{\boldsymbol{\mathit{ x}}_1^{'}\cdot\boldsymbol{\mathit{ x}}_1^{'}}\boldsymbol{\mathit{ x}}_1^{'}\right)=\hat O\boldsymbol{\mathit{ x}}_2-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_2\cdot\mathit{ x}}_1^{'}}{\boldsymbol{\mathit{ x}}_1^{'}\cdot\boldsymbol{\mathit{ x}}_1^{'}}\hat O\boldsymbol{\mathit{ x}}_1^{'}=a\left(\boldsymbol{\mathit{ x}}_2-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_2\cdot\mathit{ x}}_1^{'}}{\boldsymbol{\mathit{ x}}_1^{'}\cdot\boldsymbol{\mathit{ x}}_1^{'}}\boldsymbol{\mathit{ x}}_1^{'}\right)$

In the presence of a third linearly independent vector $\boldsymbol{\mathit{ x}}_3$ that is nonorthogonal to $\boldsymbol{\mathit{ x}}_1$ and $\boldsymbol{\mathit{ x}}_2$ (where $\hat O\boldsymbol{\mathit{ x}}_3=a\boldsymbol{\mathit{ x}}_3$ ), the vector $\boldsymbol{\mathit{ x}}_{3a}^{'}$ that is orthogonal to $\boldsymbol{\mathit{ x}}_{1}^{'}$ is $\boldsymbol{\mathit{ x}}_{3a}^{'}=\boldsymbol{\mathit{ x}}_3-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_3\cdot\mathit{ x}}_1^{'}}{\boldsymbol{\mathit{ x}}_1^{'}\cdot\boldsymbol{\mathit{ x}}_1^{'}}\boldsymbol{\mathit{ x}}_1^{'}$ (c.f. eq111). To determine the component of $\boldsymbol{\mathit{ x}}_3$ that is orthogonal to both $\boldsymbol{\mathit{ x}}_{1}^{'}$ and $\boldsymbol{\mathit{ x}}_{2}^{'}$ , let $\boldsymbol{\mathit{ x}}_{3}^{'}$ be the component of $\boldsymbol{\mathit{ x}}_{3a}^{'}$ that is orthogonal to $\boldsymbol{\mathit{ x}}_{2}^{'}$ :
$\boldsymbol{\mathit{ x}}_3^{'}=\boldsymbol{\mathit{ x}}_{3a}^{'}-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_{3a}^{'}\cdot\mathit{ x}}_2^{'}}{\boldsymbol{\mathit{ x}}_2^{'}\cdot\boldsymbol{\mathit{ x}}_2^{'}}\boldsymbol{\mathit{ x}}_2^{'}$

We can immediately see that $\boldsymbol{\mathit{ x}}_{3}^{'}$ is orthogonal to both $\boldsymbol{\mathit{ x}}_{1}^{'}$ and $\boldsymbol{\mathit{ x}}_{2}^{'}$ because it is sum of two vectors $\boldsymbol{\mathit{ x}}_{3a}^{'}$ and $\boldsymbol{\mathit{ x}}_{2}^{'}$ , each of which is orthogonal to $\boldsymbol{\mathit{ x}}_{1}^{'}$ (and hence the dot product $\boldsymbol{\mathit{ x}}_{3}^{'}\cdot\boldsymbol{\mathit{ x}}_{1}^{'}$ is zero). Substituting $\boldsymbol{\mathit{ x}}_{3a}^{'}=\boldsymbol{\mathit{ x}}_3-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_3\cdot\mathit{ x}}_1^{'}}{\boldsymbol{\mathit{ x}}_1^{'}\cdot\boldsymbol{\mathit{ x}}_1^{'}}\boldsymbol{\mathit{ x}}_1^{'}$ in the above equation, noting that $\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_3\cdot\mathit{ x}}_1^{'}}{\boldsymbol{\mathit{ x}}_1\cdot\boldsymbol{\mathit{ x}}_1^{'}}$ and $\frac{\boldsymbol{\mathit{ x}}_2^{'}}{\boldsymbol{\mathit{ x}}_2^{'}\cdot\boldsymbol{\mathit{ x}}_2^{'}}$ are scalars and that $\boldsymbol{\mathit{ x}}_1^{'}\cdot\boldsymbol{\mathit{ x}}_2^{'}=0$ , we have

$\boldsymbol{\mathit{ x}}_3^{'}=\boldsymbol{\mathit{ x}}_{3}-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_{3}\cdot\mathit{ x}}_1^{'}}{\boldsymbol{\mathit{ x}}_1\cdot\boldsymbol{\mathit{ x}}_1^{'}}\boldsymbol{\mathit{ x}}_1^{'}-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_{3}\cdot\mathit{ x}}_2^{'}}{\boldsymbol{\mathit{ x}}_2^{'}\cdot\boldsymbol{\mathit{ x}}_2^{'}}\boldsymbol{\mathit{ x}}_2^{'}\;\;\;\;\;\;\;\;(112)$

Therefore, for a set of three linearly independent nonorthogonal vectors $\{\boldsymbol{\mathit{ x}}_1,\boldsymbol{\mathit{ x}}_2,\boldsymbol{\mathit{ x}}_3\}$ , the transformed set of vectors, which are orthogonal to one another, is $\{\boldsymbol{\mathit{ x}}_1^{'},\boldsymbol{\mathit{ x}}_2^{'},\boldsymbol{\mathit{ x}}_3^{'}\}$ , where

$\boldsymbol{\mathit{ x}}_1^{'}=\boldsymbol{\mathit{ x}}_{1}$

$\boldsymbol{\mathit{ x}}_2^{'}=\boldsymbol{\mathit{ x}}_{2}-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_{2}\cdot\mathit{ x}}_1^{'}}{\boldsymbol{\mathit{ x}}_1^{'}\cdot\boldsymbol{\mathit{ x}}_1^{'}}\boldsymbol{\mathit{ x}}_1^{'}$

For a set of linearly independent nonorthogonal vectors $\{\boldsymbol{\mathit{ x}}_1,\boldsymbol{\mathit{ x}}_2,\cdots,\boldsymbol{\mathit{ x}}_k\}$ , the transformed set of vectors, which are orthogonal to one another, is $\{\boldsymbol{\mathit{ x}}_1^{'},\boldsymbol{\mathit{ x}}_2^{/},\cdots,\boldsymbol{\mathit{ x}}_k^{'}\}$ with the k-th transformed vector as

$\boldsymbol{\mathit{ x}}_k^{'}=\boldsymbol{\mathit{ x}}_{k}-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_{k}\cdot\mathit{ x}}_1^{'}}{\boldsymbol{\mathit{ x}}_1\cdot\boldsymbol{\mathit{ x}}_1^{'}}\boldsymbol{\mathit{ x}}_1^{'}-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_{k}\cdot\mathit{ x}}_2^{'}}{\boldsymbol{\mathit{ x}}_2^{'}\cdot\boldsymbol{\mathit{ x}}_2^{'}}\boldsymbol{\mathit{ x}}_2^{'}-\cdots-\frac{\boldsymbol{\boldsymbol{\mathit{ x}}_{k}\cdot\mathit{ x}}_{k-1}^{'}}{\boldsymbol{\mathit{ x}}_{k-1}^{'}\cdot\boldsymbol{\mathit{ x}}_{k-1}^{'}}\boldsymbol{\mathit{ x}}_{k-1}^{'}$

The corresponding orthonormal vectors are $\boldsymbol{\mathit{ e}}_1=\frac{\boldsymbol{\mathit{ x}}_1^{'}}{\vert\boldsymbol{\mathit{ x}}_1^{'}\vert}$ , $\boldsymbol{\mathit{ e}}_2=\frac{\boldsymbol{\mathit{ x}}_2^{'}}{\vert\boldsymbol{\mathit{ x}}_2^{'}\vert}$ , … , $\boldsymbol{\mathit{ e}}_k=\frac{\boldsymbol{\mathit{ x}}_k^{'}}{\vert\boldsymbol{\mathit{ x}}_k^{'}\vert}$ . An example of the application of the Gram-Schmidt process is the orthogonalisation of nonorthogonal Slater-type orbitals in the Hartree-Fock method.

In general, the Gram-Schmidt process applies to any mathematical objects that form a vector space equipped with an inner product. A set of matrices can be treated as vectors within a matrix vector space because matrices of the same size satisfy the vector-space rules under addition and scalar multiplication. Furthermore, the inner product of two same-sized complex matrices, $A$ and $B$ , can be defined using the Frobenius inner product:

$\langle A\vert B\rangle=Tr(A^{\dagger}B)$

For example, let $A$ and $B$ be two linearly independent, non-orthogonal matrices in the Lie algebra $\mathfrak{su}(2)$ :

$A=\begin{pmatrix}i&0\\0&-i\end{pmatrix}\;\;\;\;\;\;\;\;B=\begin{pmatrix}i&1\\-1&-i\end{pmatrix}$

They are not orthogonal because their inner product is non-zero:

$Tr(A^{\dagger}B)=Tr\biggr\[\begin{pmatrix}-i&0\\0&i\end{pmatrix}\begin{pmatrix}i&1\\-1&-i\end{pmatrix}\biggr\]=Tr\begin{pmatrix}i&-i\\-i&1\end{pmatrix}=2$

Let $A=A'$ , where $\langle A'\vert A'\rangle=Tr(A^{\dagger}A)=\begin{pmatrix}1&0\\0&1\end{pmatrix}=2$ . To obtain a new matrix $B'$ orthogonal to $A'$ , we apply eq111:

$B'=B-\frac{\langle B\vert A'\rangle}{\langle A'\vert A'\rangle}A'=\begin{pmatrix}i&1\\-1&-i\end{pmatrix}-\frac{Tr(B^{\dagger}A)}{2}\begin{pmatrix}i&0\\0&-i\end{pmatrix}=\begin{pmatrix}0&1\\-1&0\end{pmatrix}$

The resulting matrices are orthogonal:

$\langle A'\vert B'\rangle=Tr(A^{\dagger}B')=Tr\biggr\[\begin{pmatrix}-i&0\\0&i\end{pmatrix}\begin{pmatrix}0&1\\-1&0\end{pmatrix}\biggr\]=Tr\begin{pmatrix}0&-i\\-i&0\end{pmatrix}=0$

To normalise the orthogonalised matrices to form an orthonormal set, we divide each matrix by its Frobenius norm: $\|M\|_F=\sqrt{\langle M\vert M\rangle}$ . Since $\|A'\|_F=\sqrt{\langle A'\vert A'\rangle}=\sqrt{2}$ and $\|B'\|_F=\sqrt{\langle B'\vert B'\rangle}=\sqrt{2}$ , the corresponding orthonormal matrices are

$A'=\frac{1}{\sqrt{2}}\begin{pmatrix}i&0\\0&-i\end{pmatrix}\;\;\;\;\;\;\;\;B'=\frac{1}{\sqrt{2}}\begin{pmatrix}0&1\\-1&0\end{pmatrix}$

These satisfy $\langle A'\vert A'\rangle=\langle B'\vert B'\rangle=1$ and $\langle A'\vert B'\rangle=0$ , forming an orthonormal set in $\mathfrak{su}(2)$ .

Gram-Schmidt Process

Next article: Spin-orbital

Previous article: Canonical Hartree-Fock equations

Content page of computational chemistry

Content page of advanced chemistry

Main content page

Leave a Reply Cancel reply