An elegant algebraic identity says

(a^2 + b^2)*c^2 + d^2) = (ac + bd)^2 + (ad - bc)^2

If x is the vector [a b] and y is the vector [c d] then this identity can be written

begin{vmatrix} a & b \ c & d end{vmatrix}^2 = begin{vmatrix} xcdot x & x cdot y \ xcdot y & y cdot y end{vmatrix}

where the dot indicates the usual dot product. I posted this on Twitter the other day.

Gram matrix

Now suppose that x and y are vectors of any length n. The matrix on the right hand side above is called the Gram matrix of x and y, and its determinant is called the Gram determinant or Gramian.

Correlation

Let θ be the angle between x and y in ℝn. Then

costheta = frac{langle x , yrangle} {||x|| , || y ||}

where ⟨x, y⟩ is the dot product of x and y.

If x and y are data vectors, then cos θ is their correlation.

The Gram matrix G can be written in terms of cos θ:

G = begin{pmatrix} langle x, xrangle & langle x ,yrangle \ langle x, yrangle & langle y ,yrangle end{pmatrix} = ||x||^2 begin{pmatrix} 1 & cos theta \ cos theta & 1 end{pmatrix}

Generalization

The idea of the Gram matrix generalizes to more than two vectors. If we have m vectors, the Gram matrix is m × m whose (i, j) entry is the dot product of the ith and jth vectors.

Note that the dimension n of the vectors does not have to equal the dimension m of the Gram matrix.

If m does equal n, then we have the theorem that the square of the matrix whose rows are the vectors xi equals the Gram matrix. If

x_i = begin{bmatrix}x_i^1 & x_i^2 & x_i^3 & cdots & x_i^n end{bmatrix}

then we can state this as

begin{bmatrix} x_1^1 & x_1^2 & x_1^3 & cdots & x_1^n \ x_2^1 & x_2^2 & x_2^3 & cdots & x_2^n \ x_3^1 & x_3^2 & x_3^3 & cdots & x_3^n \ vdots & vdots & vdots & cdots & vdots \ x_n^1 & x_n^2 & x_n^3 & cdots & x_n^n \ end{bmatrix}^2 = begin{bmatrix} langle x_1 , x_1 rangle & langle x_1 , x_2rangle & langle x_1, x_3 rangle & cdots & langle x_1, x_n rangle \ langle x_2 , x_1 rangle & langle x_2 , x_2rangle & langle x_2, x_3 rangle & cdots & langle x_2, x_n rangle \ langle x_3 , x_1 rangle & langle x_3 , x_2rangle & langle x_3, x_3 rangle & cdots & langle x_3, x_n rangle \ vdots & vdots & vdots & cdots & vdots \ langle x_n , x_1 rangle & langle x_n , x_2rangle & langle x_n, x_3 rangle & cdots & langle x_n, x_n rangle \ end{bmatrix}

If m does not equal n then the two matrices cannot be equal because they have different dimensions, but the determinants of the two sides are equal. That is, the square of the m-dimensional volume of the span of the x‘s inside ℝn equals their Gram determinant.

The Gram determinant can be defined for more general inner products than the dot product in ℝn. It could be, for example, the integral of the product of two functions.

Related posts

The post Gram matrix first appeared on John D. Cook.

Previous Post
Next Post