Glosarium

Pilih salah satu kata kunci di sebelah kiri…

Linear AlgebraTransposes

Waktunya membaca: ~25 min

The dot product gives us a compact way to express the formula for an entry of a matrix product: to obtain the $(i,j)$th entry of a matrix product AB, we dot the $i$th row of A and the $j$th column of B.

However, the matrix product by itself is not quite flexible enough to handle a common use case: suppose we have two matrices A and B which contain tabular data stored in the same format. For example, suppose that the columns of A store the vectorized word counts for a series of emails sent from Alice, while B stores vectorized word counts for a series of emails sent from Bob. If we want to calculate the similarity of each of Alice's email to each of Bob's emails, then we want to dot the columns of A—not its rows—with the columns of B.

So we define the transpose A' of a matrix A to be the matrix which results from switching the rows and columns of A.

Definition
If A is an m\times n matrix, then its transpose A' is defined to be the matrix with n rows whose i th row is equal to the i th column of A, for each i from 1 to n.

Example
If A = \begin{bmatrix} 1 & 2 & 3 \\\ 4 & 5 & 6 \end{bmatrix}, then A' =\begin{bmatrix} 1 & 4 \\\ 2 & 5 \\\ 3 & 6 \end{bmatrix}.

With this definition in hand, we can write the matrix whose entries are the dot products of columns of A and B as A' B.

Transpose properties

Let's develop a few properties of the transpose so that we can manipulate matrix expressions involving the transpose. First, we note that the transpose is a linear operator, meaning that (cA+B)' = cA' + B' whenever c is a constant and A and B are matrices.

Taking the transpose also interacts nicely with matrix multiplication:

Exercise
Suppose that A is an m \times n matrix and that B is an n \times p matrix. Exactly one of the following expressions is equal to (AB)' in general—identify the correct answer choice by checking the dimensions of each matrix in each expression.

Confirm your conjecture numerically in Python with some random matrices. You can generate a random m \times n matrix using np.random.random_sample((m,n)), the transpose of A is accessed as A.T, and the product of A and B is A * B.

Confirm your conjecture numerically in Julia with some random matrices. You can generate a random m \times n matrix using rand(m, n).

import numpy as np

Solution. Since AB is an m \times p matrix, the matrix (AB)' is p \times m.

  • A' B': this is an n \times m matrix multiplied by a p \times n matrix, and if m \neq p it is not defined. If it is defined, it gives an n \times n matrix.
  • B' A': this is a p \times n matrix multiplied by an n \times m matrix and hence is a p \times m matrix.
  • AB is an m \times p matrix, and A' is an n \times m matrix. If p \neq n this is not defined. If it is defined, it gives an m \times m matrix.

We see that the only matrix product that is always defined, and in fact gives the right dimensions, is the second option. And in fact, we have

\begin{align*}(AB)' = B' A'\end{align*}

in general.

The following block of code checks the equation for a particular random example.

import numpy as np
A = np.random.random_sample((3,7))
B = np.random.random_sample((7,3))
np.allclose((A @ B).T, B.T @ A.T)
A = rand(3, 7)
B = rand(7, 3)
isapprox((A * B)', B' * A')

Symmetric Matrices

In some applications, a matrix will have the property that its $(i,j)th entry is necessarily equal to its(j,i)$th entry. For example, suppose we have an ordered list of 100 cell phone towers, and we define the 100 \times 100 matrix whose $(i,j)$th entry is equal to the distance from tower i to tower j. Such a matrix is said to be symmetric.

Definition
If A is an n\times n matrix satisfying the equation A = A', we say that A is symmetric.

Exercise
Suppose that A is a symmetric matrix, B is a matrix, and c \in \mathbb{R}. Which of the following is necessarily equal to (c^2 (A+B)' + A)'?

XEQUATIONX3242XEQUATIONX
XEQUATIONX3243XEQUATIONX
XEQUATIONX3244XEQUATIONX
XEQUATIONX3245XEQUATIONX
XEQUATIONX3246XEQUATIONX

Solution. We have

\begin{align*}(c^2 (A+B)' + A)' & = (c^2 (A' + B') + A)' \\ & = (c^2 A' + c^2 B' + A)' \\ & = c^2 (A')' + c^2 (B')' + A' \\ & = c^2 A + c^2 B + A' \\ & = c^2 A + c^2 B + A' \\ & = (c^2 + 1)A + c^2 B\end{align*}

Here we used that (X')' = X for any matrix X, and that A' = A for a symmetric matrix A. This leaves (3) as the correct answer. (5) is close, but incorrect if B \neq B'.

In the case where A is a n \times 1 matrix and B is an n\times 1 for some n, then A' B is a 1 \times 1 matrix, which we may think of as just a number. This means that if \mathbf{x} and \mathbf{y} are vectors in \mathbb{R}^n then the dot product \mathbf{x} \cdot \mathbf{y} may be written as \mathbf{x}' \mathbf{y}. This representation can be useful for manipulating expressions involving dot products:

Exercise
Show that

\begin{align*}\mathbf{u} \cdot (A\mathbf{v}) = (A'\mathbf{u})\cdot \mathbf{v}\end{align*}

for all m\times n matrices A and all vectors \mathbf{u} \in \mathbb{R}^m and \mathbf{v} \in \mathbb{R}^n.

Solution. Since \left(A'\right)' = A, we have

\begin{align*}\left(A' \mathbf{u}\right) \cdot \mathbf{v} &= \left(A' \mathbf{u}\right)' \mathbf{v} \\ &= \mathbf{u}' \left(A'\right)' \mathbf{v} \\ &= \mathbf{u}' \left(A\mathbf{v}\right) \\ &= \mathbf{u} \cdot \left(A\mathbf{v}\right).\end{align*}

In other words, we may move a matrix which is pre-multiplying one of the vectors in a dot product to the other vector, at the cost of taking its transpose. Let's look at one application of this property whose importance will be explored in subsequent sections.

Exercise
Show that \mathbf{x}\cdot (A' A\mathbf{x}) \geq 0 for all m\times n matrices A and all \mathbf{x} \in \mathbb{R}^n.

Solution. We have

\begin{align*}\mathbf{x}\cdot (A' A\mathbf{x}) = ((A')'\mathbf{x})\cdot (A\mathbf{x}) = (A\mathbf{x}) \cdot (A\mathbf{x}) = |A \mathbf{x}|^2 \geq 0.\end{align*}

Bruno
Bruno Bruno