# 14 Pre-Class Assignment: Fundamental Spaces¶ ## 1. Orthogonal Complement¶

Definition: A vector $$u$$ is orthogonal to a subspace $$W$$ of $$R^n$$ if $$u$$ is orthogonal to any $$w$$ in $$W$$ ($$u\cdot w=0$$ for all $$w\in W$$).

For example, consider the following figure, if we consider the plane to be a subspace then the perpendicular vector comming out of the plane is is orthoginal to any vector in the plane: Definition: The orthogonal complement of $$W$$ is the set of all vectors that are orthogonal to $$W$$. The set is denoted as $$W_{\bot}$$.

QUESTION: Is $$W_\bot$$ a subspace of $$R^n$$? Justify your answer briefly.

QUESTION: What are the vectors in both $$W$$ and $$W_\bot$$?

from IPython.display import YouTubeVideo


### Projection of a Vector onto a Subspace¶

Think of a projection onto a subspace is analogous to a shadow on a surface. Aspects of an objects 3D space is represented in a 2D shadow but you can’t take the shadow by itself and exactly recreate the 3D surface. Image from https://commons.wikimedia.org

The following is the matimatical defination of projection onto a subspace.

Definition: Let $$W$$ be a subspace of $$R^n$$ of dimension $$m$$. Let $$\{w_1,\cdots,w_m\}$$ be an orthonormal basis for $$W$$. Then the projection of vector $$v$$ in $$R^n$$ onto $$W$$ is denoted as $$\mbox{proj}_Wv$$ and is defined as $$$\mbox{proj}_Wv = (v\cdot w_1)w_1+(v\cdot w_2)w_2+\cdots+(v\cdot w_m)w_m$$$

Another way to say the above defination is that the project of $$v$$ onto the $$W$$ is just the sumation of $$v$$ projected onto each vector in a basis of $$W$$

Remarks:

Recall in the lecture on Projections, we discussed the projection onto a vector, which is the case for $$m=1$$. We used the projection for $$m>1$$ in the Gram-Schmidt algorithm.

The projection does not depend on which orthonormal basis you choose.

If $$v$$ is in $$W$$, we have $$\mbox{proj}_Wv=v$$.

### The Orthogonal Decomposition Theorem¶

Theorem: Let $$W$$ be a subspace of $$R^n$$. Every vector $$v$$ in $$R^n$$ can be written uniquely in the form $$$v= w+w_{\bot},$$$$where$$w$$is in$$W$$and$$w_\bot$$is orthogonal to$$W$$(i.e.,$$w_\bot$$is in$$W_\bot$$). In addition,$$w=\mbox{proj}Wv$$, and$$w\bot = v-\mbox{proj}_Wv$.

Definition: Let $$x$$ be a point in $$R^n$$, $$W$$ be a subspace of $$R^n$$. The distance from $$x$$ to $$W$$ is defined to be the minimum of the distances from $$x$$ to any point $$y$$ in $$W$$. $$$d(x,W)=\min \{\|x-y\|: \mbox{ for all }y \mbox{ in } W\}.$$$$The optimal$$y$$can be achieved at$$\mbox{proj}_Wx$$, and$$d(x,W)=|x-\mbox{proj}_Wx|$.

QUESTION: Let $$v=(3, 2, 6)$$ and $$W$$ is the subspace consisting all vectors with the form $$(a, b, b)$$. Find the projection of $$v$$ onto $$W$$.

QUESTION: Let $$v=(3, 2, 6)$$ and $$W$$ is the subspace consisting all vectors with the form $$(a, b, b)$$. Find the distance from $$v$$ to $$W$$.

## 2. The Four Fundamental Spaces¶

In the lecture on Change Basis, we talked about four subspaces based on a matrix $$A$$:

Row space of $$A$$: linear combination of all rows of $$A$$

Column space of $$A$$: linear combination of all columns of $$A$$

Null space or kernel of $$A$$: all $$x$$ such that $$Ax=0$$

Null space of $$A^\top$$: all $$y$$ such that $$A^\top y =0$$

In this course we represent a system of linear equations as $$Ax=b$$. The matrix $$A$$ can be viewed as taking a point $$x$$ in the input space and projecting that point to $$b$$ in the output space.

It turns out, everything we need to know about $$A$$ is represented by four fundamental vector spaces. Two of the four spaces are easily defined as follows:

Row space of $$A$$: linear combination of all rows of $$A$$

Column space of $$A$$: linear combination of all columns of $$A$$

The other two fundamental spaces are defined by a concept called the Null Space. The Null space is calculated by finding all the solutions to the homogeneous system $$Ax=0$$. The final two fundamental spaces are defined as follows:

Null space or kernel of $$A$$: all $$x$$ such that $$Ax=0$$

Null space of $$A^\top$$: all $$y$$ such that $$A^\top y =0$$

## 3. Independent Learning¶

DO THIS: Find a YouTube video that helps you understand the four fundamental spaces.

QUESTION: What is the URL for your video?

DO THIS: Add the link to the video to the code below. Try embedding the link in the provided Python YouTubeVideo Function by replacing XXXXX with the video ID.

from IPython.display import YouTubeVideo


QUESTION: What criteria did you use in selecting your video?

QUESTION: How long into a video did you go before deciding if it was good or bad?

QUESTION: What did you like about the video you selected.

QUESTION: What didn’t you like about the video?

## 4. Assignment_wrap-up¶

Assignment-Specific QUESTION: What is the URL for your video for the four Fundamental spaces?

QUESTION: Summarize what you did in this assignment.

QUESTION: What questions do you have, if any, about any of the topics discussed in this assignment after working through the jupyter notebook?

QUESTION: How well do you feel this assignment helped you to achieve a better understanding of the above mentioned topic(s)?

QUESTION: What was the most challenging part of this assignment for you?

QUESTION: What was the least challenging part of this assignment for you?

QUESTION: What kind of additional questions or support, if any, do you feel you need to have a better understanding of the content in this assignment?

Written by Dr. Dirk Colbry, Michigan State University 