Length of projection of vector $v$ to $u$
Let $u$ be a unit vector in $n$ dimensional space. If I have another vector $v$ and want to project it on a line parallel to vector $u$, what is the formula for calculating the distance between point of projection and the origin?
The only thing I've managed to find is that it's equal to dot product $u \cdot v$ or equivalently $u^T v$, but I don't why that is. How to justify it?
$\endgroup$ 33 Answers
$\begingroup$As noted in the comments, the geometric interpretation of the dot product works well in $\mathbb{R}^2$.
I want to sum here this result.
We have two vectors $\mathbf{v}= v_1\mathbf{e_1}+v_2\mathbf{e_2}$ and $\mathbf{u}= u_1\mathbf{e_1}+u_2\mathbf{e_2}$ and, being $\{\mathbf{e_1},\mathbf{e_2}\}$ the canonical basis we know that: $$ v_1=|\mathbf{v}|\cos \alpha \quad v_2=|\mathbf{v}|\sin \alpha $$ and
$$ u_1=|\mathbf{u}|\cos \beta \quad u_2=|\mathbf{u}|\sin \beta $$
where $\alpha, \beta$ are the angles between $\mathbf{e_1}$ and the two vectors.
Now we define :
$\mathbf{u}\cdot\mathbf{v}=|\mathbf{u}||\mathbf{v}|\cos\theta$
where $\theta=\beta-\alpha$ is the angle between two vectors. Using trigonometry we find: $$ |\mathbf{u}||\mathbf{v}|\cos(\beta-\alpha)=|\mathbf{u}||\mathbf{v}|(\cos \alpha \cos \beta+\sin\alpha \sin \beta)= $$ $$ =(|\mathbf{v}|\cos \alpha) (|\mathbf{u}|\cos\beta)+(|\mathbf{v}|\sin \alpha)(|\mathbf{u}|\sin \beta)= $$ $$ =u_1v_1+u_2v_2 $$
This is fine and give us the machinery to find angles between vectors an to project a vector over an other vector using the components.
Generalize this result to any Hilbert space is not so simple, and we have to use the Cauchy-Schwarz inequality that states:
$$ \left|\sum_{i=1}^n{x_iy_i} \right|^2\le\sum_{i=1}^n{x_i^2}\sum_{i=1}^n{y_i^2} $$
So, given $\mathbf{x}=\sum_{i=1}{x_i\mathbf{e_i}}$ and $\mathbf{y}=\sum_{i=1}{y_i\mathbf{e_i}}$ we can define the dot product as: $$ \mathbf{x}\cdot \mathbf{y}=\sum_{i=1}{x_i y_i} $$ and we have: $ \mathbf{x}\cdot \mathbf{y}\le ||\mathbf{x}||\; ||\mathbf{y}|| $ So we see that there exists an angle $\theta$ such that
$$ \cos \theta = \dfrac{\mathbf{x}\cdot \mathbf{y}}{ ||\mathbf{x}||\; ||\mathbf{y}||} $$ and we can define such $\theta$ as the angle between the two vectors.
And , finally, from this we have an analogy with the $\mathbb{R}^2$ situation ad we can find the prjection of a vector on another one.( And this work also for infinite dimensional Hilbet spaces).
$\endgroup$ 2 $\begingroup$In general: $${\rm proj}_{\bf u}{\bf v} = \frac{{\bf u}\cdot {\bf v}}{{\bf u}\cdot{\bf u}}{\bf u}.$$ Here $\|{\bf u}\| = 1$, so ${\bf u}\cdot {\bf u} = 1$ and we have ${\rm proj}_{\bf u}{\bf v} = ({\bf u}\cdot{\bf v}){\bf u}$. Hence the distance to the origin is: $$\|{\rm proj}_{\bf u}{\bf v} - {\bf 0}\| = \|{\rm proj}_{\bf u}{\bf v}\| = \|({\bf u}\cdot{\bf v}){\bf u}\| = |{\bf u}\cdot{\bf v}|\|{\bf u}\| = |{\bf u}\cdot{\bf v}|.$$
$\endgroup$ 2 $\begingroup$Say the length of projection is $l$, then the projection vector is $l\mathrm{u}$
Next notice that $\mathrm{u}$ is perpendicular to $\mathrm{v}-l\mathrm{u}$, therefore the dot product has to be $0$ $$\mathrm{u}^T(\mathrm{v}-l\mathrm{u}) = 0 $$
Solving for $l$ we get $$l = \dfrac{\mathrm{u}^T\mathrm{v}}{\mathrm{u}^T\mathrm{u}}$$
as desired.
$\endgroup$
