Notebook 11 – Math 2121, Fall 2020

In practice, (abstract) vector spaces that are not readily identified with subspaces of ${\mathbb{R}}^n$ often arise as function spaces, like the vector space of functions $\mathbb{R}\to \mathbb{R}$. Let's examine how we can think of vectors, vector addition, and scalar multiplication in such spaces.

Recall from a few weeks ago how we can visualize vector operations in ${\mathbb{R}}^2$.

First, create some vectors.

Now choose some scalars.

a =

2

b =

-1.9

Visualizing the vector space operations for

$$\mathsf{F}\mathsf{u}\mathsf{n}(\mathbb{R},\mathbb{R})=\{\text{functions }f:\mathbb{R}\to \mathbb{R}\}$$

is not so different.

Create some functions $\mathbb{R}\to \mathbb{R}$.

The sinc function is defined by $\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{c}(x)=\frac{\sin (\pi x)}{\pi x}$.

You may remember from calculus that $\underset{x\to 0}{lim}\frac{\sin (x)}{x}=1$.

The sinc function is therefore defined at all $x\in \mathbb{R}$ and has $\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{c}(0)=1$.

Choose some scalars in $\mathbb{R}$:

a =

2.7

b =

-4.1

The function space $\mathsf{F}\mathsf{u}\mathsf{n}(\mathbb{R},\mathbb{R})=\{\text{functions }f:\mathbb{R}\to \mathbb{R}\}$ is really big, too big for many natural operations like differentiation and integration to make sense for all elements.

The space $\mathsf{F}\mathsf{u}\mathsf{n}(\mathbb{R},\mathbb{R})$ contains many natural subspaces that may be more suitable to applications:

• The space $L^1(\mathbb{R})=\{f\in \mathsf{F}\mathsf{u}\mathsf{n}(\mathbb{R},\mathbb{R}):{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}|f(x)|dx<\mathrm{\infty }\}$ of integrable functions.

• The space $L^2(\mathbb{R})=\{f\in \mathsf{F}\mathsf{u}\mathsf{n}(\mathbb{R},\mathbb{R}):f^2\in L^1(\mathbb{R})\}$ of square-integrable functions.

• The space $L^{\mathrm{\infty }}(\mathbb{R})=\{f\in \mathsf{F}\mathsf{u}\mathsf{n}(\mathbb{R},\mathbb{R}):\mathrm{\exists }N\text{ with }|f(x)|<N\mathrm{\forall }x\}$ of bounded functions.

Let's think about linear transformations between these spaces.

A linear transformation $T:L^2(\mathbb{R})\to L^2(\mathbb{R})$ is a function that takes a square-integrable function $f$ as input and gives another square-integrable function $T(f)$ as output, such that

$$T(f+g)=T(f)+T(g)\text{and}T(cf)=cT(f)$$

for all $f,g\in L^2(\mathbb{R})$ and $c\in \mathbb{R}$.

Perhaps the most famous linear transformation $L^2(\mathbb{R})\to L^2(\mathbb{R})$ is the Fourier transform $\mathsf{F}\mathsf{T}$.

One definition of this (technically, the Fourier cosine transform) is

$$\mathsf{F}\mathsf{T}(f)=(\text{the function with formula }x\mapsto {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}f(y)\cos (2\pi xy)dy).$$

This transformation is linear because of the rules for integration from calculus, which allow us to expand sums and scalar multiples of functions under the integral sign.

Let's see how we can implement $\mathsf{F}\mathsf{T}$ in Julia and explore some examples.

Our definition of the Fourer transform does not make sense for $\cos (x)$, since this function is not square-integrable: ${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}|\cos (x){|}^{2}dx=\mathrm{\infty }$.

But we can apply $\mathsf{F}\mathsf{T}$ to things like the truncated cosine function

$$\cos (x)1_{[-2\pi ,2\pi ]}(x)$$

where $1_{[a,b]}(x)$ is the function that takes values $1$ for $a\le b\le x$ and $0$ for all other $x$.

The result is the sum of two 'pulses' at $x=\pm \frac{1}{2\pi }$.

What about the Fourier transform of $\cos (x^2)1_{[-4{\pi }^2,4{\pi }^2]}(x)$?

Ignoring error, the Fourier transform of the above function is exactly $\sqrt{\pi }\cos ({\pi }^2{x}^2-\pi /4)$.

An interesting example comes $\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{c}(x)$, whose Fourier transform looks exactly like $1_{[-\frac{1}{2},\frac{1}{2}]}(x)$.

The Fourier transform of $1_{[-\frac{1}{2},\frac{1}{2}]}(x)$, in turn, looks just like $\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{c}(x)$.

We can use these examples to construct eigenvectors of $\mathsf{F}\mathsf{T}$.

If $T:V\to V$ is a linear transformation from a vector space to itself, then an eigenvector is a nonzero element $v\in V$ such that $T(v)=\lambda v$ for some scalar $\lambda \in \mathbb{R}$.

In this case $\lambda$ is the corresponding eigenvalue.

Now, if $\mathsf{F}\mathsf{T}(f)=g$ and $\mathsf{F}\mathsf{T}(g)=f$ where $f\ne g$, then by linearity

$$\mathsf{F}\mathsf{T}(f+g)=\mathsf{F}\mathsf{T}(f)+\mathsf{F}\mathsf{T}(g)=g+f=f+g$$

so $f+g$ is an eigenvector with eigenvalue $1$. Similarly,

$$\mathsf{F}\mathsf{T}(f-g)=\mathsf{F}\mathsf{T}(f)-\mathsf{F}\mathsf{T}(g)=g-f=-(f-g)$$

so $f-g$ is an eigenvector with eigenvalue $-1$.

We can apply this to $f(x)=\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{c}(x)$ and $g(x)=1_{[-\frac{1}{2},\frac{1}{2}]}(x)$.

This shows that $\lambda =-1$ and $\lambda =1$ are both eigenvalues of $\mathsf{F}\mathsf{T}$.

Here are some other interesting eigenvectors of $\mathsf{F}\mathsf{T}$ for these eigenvalues:

Are there are any other eigenvalues $\lambda$ for $\mathsf{F}\mathsf{T}$ besides $\lambda =1$ and $\lambda =-1$? Yes, $\lambda =0$:

The numbers $-1,0,1$ are the only eigenvalues of $\mathsf{F}\mathsf{T}$. Here's why.

A function $f:\mathbb{R}\to \mathbb{R}$ is even if $f(-x)=f(x)$ for all $x$ and odd if $f(-x)=-f(x)$ for all $x$.

If $f\in L^2(\mathbb{R})$ is odd then $\mathsf{F}\mathsf{T}(f)=0$, as in the example above.

If $f\in L^2(\mathbb{R})$ is even then $\mathsf{F}\mathsf{T}(\mathsf{F}\mathsf{T}(f))=f$, as we saw with $f(x)=\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{c}(x)$.

Every $f\in L^2(\mathbb{R})$ can be written as $f=f_{\mathrm{e}\mathrm{v}\mathrm{e}\mathrm{n}}+f_{\mathrm{o}\mathrm{d}\mathrm{d}}$ where

$$f_{\mathrm{e}\mathrm{v}\mathrm{e}\mathrm{n}}(x)=\frac{f(x)+f(-x)}{2}\text{and}{f}_{\mathrm{o}\mathrm{d}\mathrm{d}}(x)=\frac{f(x)-f(-x)}{2}.$$

Also $\mathsf{F}\mathsf{T}(f_{\mathrm{e}\mathrm{v}\mathrm{e}\mathrm{n}})=\mathsf{F}\mathsf{T}(f{)}_{\mathrm{e}\mathrm{v}\mathrm{e}\mathrm{n}}$.

This means $\mathsf{F}\mathsf{T}(f)=\mathsf{F}\mathsf{T}(f_{\mathrm{e}\mathrm{v}\mathrm{e}\mathrm{n}})=\mathsf{F}\mathsf{T}(f{)}_{\mathrm{e}\mathrm{v}\mathrm{e}\mathrm{n}}$.

So the only way we can have $\mathsf{F}\mathsf{T}(f)=\lambda f$ is if

• the function $f$ is odd and $\lambda =0$, or

• the function $f$ is even and ${\lambda }^2=1$, since then ${\lambda }^{2}f=\mathsf{F}\mathsf{T}(\mathsf{F}\mathsf{T}(f))=f$.

Because the Fourier transform is a linear transformation $L^2(\mathbb{R})\to L^2(\mathbb{R})$, which is infinite dimensional, there is no corresponding standard matrix as there would be for a linear transformation ${\mathbb{R}}^n\to {\mathbb{R}}^m$.

As such, although we can figure out the eigenvalues of $\mathsf{F}\mathsf{T}$, it is much less straightforward to find (a basis for) the eigenvectors: we cannot do this just by finding the null space of a certain matrix.

Solving this probem is something covered in a more advanced course on Fourier/harmonic analysis.

Very briefly, why is the Fourier transform is an important/useful linear function:

• The Fourier transform converts waves to bits. More precisely, $\mathsf{F}\mathsf{T}$ converts (decaying) sinusoidal signals to discrete signals. For this reason, $\mathsf{F}\mathsf{T}$ is the foundation of signal processing.

• Not so much emphasized in our discussion, but $\mathsf{F}\mathsf{T}$ is also useful in algebra because it transforms differential equations to polynomial equations, which are easier to solve.

• Finally, and signficantly: there are efficient algorithms to compute $\mathsf{F}\mathsf{T}$.