Vectors are anything that can be added (moved) and scaled (stretched) by a number (usually real). For example, points in Euclidean space, real-valued functions, etc.
Add
Addition and scaling must be compatible in the sense that \[\color{purple}1\mathbf{v} = \mathbf{v},\quad 2\mathbf{v} = \mathbf{v} + \mathbf{v},\quad 6\mathbf{v} = 3(2\mathbf{v}), \quad 2(\mathbf{u} + \mathbf{v}) = 2\mathbf{u} + 2\mathbf{v} ...\]There is a special point, the origin \(\color{purple}\mathbf{0}\), which does not move vectors, and to which vectors scale down to. In fact, multiplying by the number zero \(0\), scales any vector to the origin; the zero vector remains the same when scaled.
Each vector has its own negative vector which moves the vector to the zero point; it can be obtained by scaling it by \(-1\). Multiplying a vector by a negative number is the same as multiplying the negative vector by the positive number: \[\color{purple}\mathbf{v} + (-\mathbf{v}) = \mathbf{0}, t(-\mathbf{v}) = (-t)\mathbf{v}\]
Dimension
0-D
The simplest vector space is the single point \(\mathbf{0}\) by itself.
1-D
A one-dimensional vector space is obtained by scaling a single non-zero vector by all the numbers; it is a line, passing through zero.
2-D
Stretching and adding two non-zero vectors that are independent (i.e., not on one line) gives a whole plane. If we call the independent vectors \(\mathbf{i}\) and \(\mathbf{j}\), then for any vector in their plane, \(v = x \mathbf{i} + y \mathbf{j}\), or, writing just the components for short, \(v = (x,y)\).
3-D
Three independent vectors give a 3-D vector space; each vector has three coordinates \((x,y,z)\).
The maximum number of independent vectors is called the dimension of the vector space; every vector can be written in terms of these basis vectors.
Abstract spaces can have 4 or more dimensions, even infinite; they cannot be drawn like the others, but the components of a vector can be shown as a sequence, or more generally, a function.
Parallel Subspaces
Parallel subspaces (lines, planes, etc.) are translated from their original position by adding a vector to them.
A 2-D stack of parallel lines filling 3-D space.
In general, the whole \(n\)-D space splits up into a \((n-m)\)-dimensional stack of parallel copies of an \(m\)-D subspace.
Any two points determine one line; three points a plane or a line; \(n\) points a subspace of dimension \(n-1\) or less.
Any two lines in the plane meet at a point, unless parallel; they need not meet in 3-D space. In general two \((n-1)\)-D subspaces meet in an \((n-2)\)-D subspace unless parallel.
Three generic planes in 3-D space meet at a point; in general, a number of \((n-k_i)\)-D subspaces meet in a subspace of dimension \(n-\sum_i(n-k_i)\), unless some are parallel.
In any triangle, if one side is moved parallel to itself, it cuts the other sides in the same proportion.
Norms
A shape is convex when the line joining any two of its points lies completely inside it.
Vectors can have a distance (and so convergence) defined on them. It is usually required that
the distance between two vectors is relative, i.e., it is unchanged when both are moved by the same amount; by translating one vector to 0, the distance boils down to the length, or norm, of a vector, \(\color{purple}|u-v|\);
the norm increases in proportion with the scaling, \(\color{purple}|r v| = |r| |v|\).
A picture of the norm can be obtained by plotting only the vectors with norm 1 — it can be any bounded convex shape which is symmetric about the origin. The norm can "depend" on the vector direction.
For example, in the plane, let the unit "disk" have the following convex diamond shape:
The lines from the red point to the black points all have the same length; so there is no unique line with the shortest distance from the point to the blue line. The triangles shown are all equilateral and have the same area, but they aren't congruent, i.e., they cannot be transformed from one to the other while preserving all lengths.
Although there are various norms on \(\mathbb{R}^n\), they all have the same convergence: a point converges when its 'shadow points' on each axis converge. This is not true in infinite dimensions.
Linear Transformations
A linear transformation is a mapping of vectors that preserves vector addition and scaling: \[\color{purple}T(\mathbf{u}+\mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v}),\qquad T(r\mathbf{v}) = r\, T(\mathbf{v})\]
Effectively, there is a superposition of each vector; in \(\mathbb{R}^2\), since \(T(\mathbf{v})=T(x\mathbf{i} + y\mathbf{j}) = x\, T(\mathbf{i}) + y\, T(\mathbf{j})\), we need only specify \(T(\mathbf{i}) = a \mathbf{i} + b \mathbf{j}\), and \(T(\mathbf{j}) = c \mathbf{i} + d \mathbf{j}\) to determine \(T\) completely; the numbers \(a, b, c, d\) are usually written in matrix form as above: \(\begin{pmatrix}a&c\\b&d\end{pmatrix}\)
The matrix \(\begin{pmatrix}1&0\\0&1\end{pmatrix}\) leaves all vectors unmoved; \(\begin{pmatrix}0&0\\0&0\end{pmatrix}\) maps all vectors to the zero vector; \(\begin{pmatrix}\lambda&0\\0&\mu\end{pmatrix}\) stretches the \(x\)-axis by \(\lambda\) and the \(y\)-axis by \(\mu\).
Notice how linear transformations map any straight line to another straight line (or to a point); and more generally, a subspace to a subspace. They also map a bounded set to another bounded set.
Most transformations are effectively the combination of scalings in different directions. These directions are called eigenvectors, and the amount of scaling is called its eigenvalue \(\lambda\), \[\color{purple}T(\mathbf{v}) = \lambda \mathbf{v}\]
Mouse over the box to see the eigenvectors of \(T\) each with the length of its eigenvalue.
For example a reflection occurs when the eigenvalue is negative, e.g. \(\begin{pmatrix}1&0\\0&-1\end{pmatrix}\). A projection is the special case when some of the eigenvalues are 0, and the rest are 1, e.g. \(\begin{pmatrix}1&0\\0&0\end{pmatrix}\); it flattens the vectors to a line. Sometimes, there is only one eigenvector and the transformation is called a shear; e.g. \(\begin{pmatrix}1&1\\0&1\end{pmatrix}\). Other times, when the transformation is a generalized rotation, there are no eigenvectors at all; in this case the eigenvalues are complex numbers; e.g. \(\begin{pmatrix}1&-1\\1&1\end{pmatrix}\).
Every linear transformation breaks up into three irreducible "pieces": a scaling, a shear, and a rotation.
Linear transformations form a ring, where addition of two transformations \(A+B\) is a superposition of their effects, and multiplication \(AB\) is the combination of the effect of one followed by the other; they can also be scaled \(rA\).
Not every matrix is invertible; in fact, there may be divisors of zero (e.g., projections), even nilpotent transformations, e.g. \(\begin{pmatrix}0.5&0.5\\-0.5&-0.5\end{pmatrix}\).
Dual Vector Spaces
A functional is a transformation that maps vectors to a straight line, i.e., to a real number \((x)\). The vectors that map to 0 form an \((n-1)\)-dimensional subspace, called the nullspace of the functional.
Example: A functional on the plane, with the white line as nullspace. Any other such functional is a stretched and rotated version of it. (Mouse over; each vector is mapped to a number: red for positives, cyan for negatives.)
Conversely, an \((n-1)\)-D subspace is in the nullspace of several functionals, that together form a dual subspace. In 3-D space, the dual space of a line is a plane, and the dual of a plane is a line. In general, the dual of an \(m\)-D subspace is \((n-m)\)-D.
Just as linear transformations act on vectors, dual transformations act on functionals.
A point can always be "separated" from a closed convex set by a functional.
Inner Products
An inner product, is a way of specifying lengths and angles between vectors; in particular which vectors are perpendicular to each other. \[\color{purple}\begin{pmatrix}a\\ b\\ c\end{pmatrix}\cdot\begin{pmatrix}x\\ y\\ z\end{pmatrix} = ax + by +cz\] The length of a vector is then \(\color{purple}|\mathbf{v}|=\sqrt{\mathbf{v}\cdot\mathbf{v}}\), and the angle between unit vectors is \(\color{purple}\angle(\mathbf{u},\mathbf{v})=\arccos(\mathbf{u}\cdot\mathbf{v})\).
The only linear transformations that preserve all angles and lengths are rotations and geometric reflections; as well as translations.Pythagoras' theorem, \(x^2+y^2=z^2\) for perpendicular vectors; also valid in infinite dimensions. Click inside to prove: the yellow area \(x^2+y^2\) remains the same, \(z^2\).
An inner-product gives the only norm whose unit vectors form an "ellipsoid", or that satisfies, for any parallelogram:sum of sides2 = sum of diagonals2
The shortest line from a point to a closed subspace is the perpendicular. As usual, points need not be literal points; here they represent images, used to correct imaging defects.
Some theorems on triangles and circles in the plane:
A triangle is determined up to translation, rotation, and reflection (congruency) by SSS, SAS, or ASA; so there is only 'one' equilateral triangle with a given side.
Any triangle can be solved using: \[\frac{a}{\sin A}=\frac{b}{\sin B}=\frac{c}{\sin C}\] \[c^2=a^2+b^2-2ab\cos C\]
The altitudes meet at a point; as do the medians, and the perpendicular bisectors; the three points are in a line. One circle goes around the six feet of the altitudes and medians.
\[\cos(A+B) =\color{purple} \cos A \cos B - \sin A \sin B\] \[\sin(A+B) = \color{#039}\sin A \cos B + \cos A \sin B\]
Bases
Vectors can be built up from basis vectors. Any unit perpendicular vectors that fill out the vector space can be used. For geometrical 3-D space, the vectors \(\color{blue}\textbf{i}, \textbf{j}, \textbf{k}\) are usually taken as a basis.
Functions are also vectors: they can be added and scaled. The Fourier series is an example of building functions using a basis of cosines and sines. Change the sequence of coefficients, by dragging their position, to build a different function.
