Kronecker product

The Kronecker product, denoted by $\otimes$ , is a multiplication method for generating a new vector space from existing vector spaces, and therefore, new vectors from existing vectors.

Consider 2 vectors spaces, e.g. $V=\mathbb{R}^{2}$ and $W=\mathbb{R}^{3}$ . For $\boldsymbol{\mathit{v}}=\begin{pmatrix} a_1\\a_2 \end{pmatrix}$ in $V$ and $\boldsymbol{\mathit{w}}=\begin{pmatrix} b_1\\b_2\\b_3 \end{pmatrix}$ in $W$ , we can define a new vector space, $V\otimes W$ , which consists of the vector $\boldsymbol{\mathit{v}}\otimes\boldsymbol{\mathit{w}}$ , where:

$\boldsymbol{\mathit{v}}\otimes\boldsymbol{\mathit{w}}=\begin{pmatrix} a_1\\a_2 \end{pmatrix}\otimes \begin{pmatrix} b_!\\b_2 \\ b_3 \end{pmatrix}=\begin{pmatrix} a_1b_1\\a_1b_2 \\a_1b_3 \\ a_2b_1 \\ a_2b_2 \\ a_2b_3 \end{pmatrix}$

If the basis vectors for $V$ and $W$ are $V=\left \{\boldsymbol{\mathit{e_1}},\boldsymbol{\mathit{e_2}}\right \}$ and $W=\left \{\boldsymbol{\mathit{f_1}},\boldsymbol{\mathit{f_2}},\boldsymbol{\mathit{f_3}}\right \}$ respectively, the basis for $V\otimes W$ is:

$V\otimes W=\{\boldsymbol{\mathit{e_1}}\otimes\boldsymbol{\mathit{f_1}},\boldsymbol{\mathit{e_1}}\otimes\boldsymbol{\mathit{f_2}},\boldsymbol{\mathit{e_1}}\otimes\boldsymbol{\mathit{f_3}},\boldsymbol{\mathit{e_2}}\otimes\boldsymbol{\mathit{f_1}},\boldsymbol{\mathit{e_2}}\otimes\boldsymbol{\mathit{f_2}},\boldsymbol{\mathit{e_2}}\otimes\boldsymbol{\mathit{f_3}}\}$

Question

Why is $V\otimes W$ a new vector space?

Answer

An $n$ -dimensional vector space is spanned by $n$ linearly independent basis vectors. The basis vectors for $V=\left \{\boldsymbol{\mathit{e_1}},\boldsymbol{\mathit{e_2}}\right \}$ and $W=\left \{\boldsymbol{\mathit{f_1}},\boldsymbol{\mathit{f_2}},\boldsymbol{\mathit{f_3}}\right \}$ are

$\boldsymbol{\mathit{e}}_1=\begin{pmatrix}1\\0\end{pmatrix}\;\;\;\boldsymbol{\mathit{e}}_2=\begin{pmatrix}0\\1\end{pmatrix}\;\;\;\boldsymbol{\mathit{f}}_1=\begin{pmatrix}1\\0\\0\end{pmatrix}\;\;\;\boldsymbol{\mathit{f}}_2=\begin{pmatrix}0\\1\\0\end{pmatrix}\;\;\;\boldsymbol{\mathit{f}}_3=\begin{pmatrix}0\\0\\1\end{pmatrix}\;\;\;$

and consequently, the basis vectors for $V\otimes W$ are

$\boldsymbol{\mathit{e}}_1\otimes\boldsymbol{\mathit{f}}_1=\begin{pmatrix}1\\0\\0\\0\\0\\0\end{pmatrix}\;\;\;\boldsymbol{\mathit{e}}_1\otimes\boldsymbol{\mathit{f}}_2=\begin{pmatrix}0\\1\\0\\0\\0\\0\end{pmatrix}\;\;\;\boldsymbol{\mathit{e}}_1\otimes\boldsymbol{\mathit{f}}_3=\begin{pmatrix}0\\0\\1\\0\\0\\0\end{pmatrix}$

$\boldsymbol{\mathit{e}}_2\otimes\boldsymbol{\mathit{f}}_1=\begin{pmatrix}0\\0\\0\\1\\0\\0\end{pmatrix}\;\;\;\boldsymbol{\mathit{e}}_2\otimes\boldsymbol{\mathit{f}}_2=\begin{pmatrix}0\\0\\0\\0\\1\\0\end{pmatrix}\;\;\;\boldsymbol{\mathit{e}}_2\otimes\boldsymbol{\mathit{f}}_3=\begin{pmatrix}0\\0\\0\\0\\0\\1\end{pmatrix}$

These 6 linearly independent basis vectors therefore span a 6-dimensional space.

This implies that $V\otimes W$ is $nm$ dimensional if $V$ is $n$ -dimensional and $W$ is $m$ -dimensional. Since $V\otimes W$ is a vector space, the vectors $\boldsymbol{\mathit{v}}\otimes\boldsymbol{\mathit{w}}$ must follow the rules of addition and multiplication of a vector space. Each vector $\boldsymbol{\mathit{v}}\otimes\boldsymbol{\mathit{w}}$ in the new vector space can then be written as a linear combination of the basis vectors $\boldsymbol{\mathit{e_i}}\otimes\boldsymbol{\mathit{f_j}}$ , i.e. $\sum c_{i,j}\boldsymbol{\mathit{e_i}}\otimes\boldsymbol{\mathit{f_j}}$ .

In general, if

$\boldsymbol{\mathit{e}}_i=\begin{pmatrix}\delta_{1i}\\\delta_{2i}\\\vdots\\\delta_{mi}\end{pmatrix}\;\;\;\;\;\boldsymbol{\mathit{f}}_j=\begin{pmatrix}\delta_{1j}\\\delta_{2j}\\\vdots\\\delta_{nj}\end{pmatrix}$ then

$\boldsymbol{\mathit{e}}_i\otimes \boldsymbol{\mathit{f}}_j=\begin{pmatrix}\delta_{1i}\delta_{1j}\\\delta_{1i}\delta_{2j}\\\vdots\\\delta_{1i}\delta_{nj}\\\delta_{2i}\delta_{1j}\\\vdots\\\delta_{mi}\delta_{nj}\end{pmatrix}$

Since the pair $(i,j)$ in $\delta_{mi}\delta_{nj}$ is distinct for each $\boldsymbol{\mathit{e_i}}\otimes\boldsymbol{\mathit{f_j}}$ vector, the Kronecker product $\boldsymbol{\mathit{e_i}}\otimes\boldsymbol{\mathit{f_j}}$ results in $m\times n$ basis vectors, which span an $m\times n$ vector space.

As mentioned in an earlier article, a vector space is a set of objects that follows certain rules of addition and multiplication. If the objects are matrices, we have a vector space of matrices. For example, the vector spaces of matrices $K=\{A,B\}$ and $L=\{C,D,E\}$ generates a new vector space of matrices $K\otimes L=\{A\otimes C,A\otimes D,A\otimes E,B\otimes C,B\otimes D,B\otimes E\}$ , where

$A\otimes C=\begin{pmatrix} a_{11}C &\cdots &a_{1n}C \\ \vdots &\ddots &\vdots \\ a_{m1}C &\cdots &a_{mn}C \end{pmatrix}$

Similarly, if the objects are functions, we have a vector space of functions $X=\{f(x),g(x),h(x),...\}$ and the Kronecker product of two vector spaces of functions $X$ and $Y$ generates a new vector space of functions $X\otimes Y$ . If $X$ and $Y$ are spanned by $m$ basis functions and $n$ basis functions respectively, $X\otimes Y$ is spanned by $mn$ basis functions.

A vector space that is generated from two separate vector spaces has applications in quantum composite systems and in group theory.

Question

What is the relation between the matrix entries of A, B and C in $A\otimes B=C$ ?

Answer

Let the matrix entries of A, B and C be $a_{ij}$ , $b_{kl}$ and $c_{pq}$ respectively, where

$\begin{matrix} i=1,\cdots ,m &j=1,\cdots ,m \\ k=1,\cdots ,n &l=1,\cdots ,n \\ p=1,\cdots ,mn &q=1,\cdots ,mn \end{matrix}$

Using the ordering convention called dictionary order, where $p$ is determined by $i$ and $k$ , and $q$ is determined by $j$ and $l$ , such that $(p,i,k)$ and $(q,j,l)$ are given by