Discrete Least Squares Approximation

Polynomial Least Squares

Given $m$ data points we can find a polynomial

$$P_n(x)=a_n{x}^n+a_{n-1}{x}^{n-1}+\cdots +a_{1}x+a_0$$

of up to degree using the least squares procedure to minimize error.

To do so, we setup a linear system of $n+1$ normal equations to solve for the $n+1$ unknown constants. These are

$$\sum _{k=0}^n{a}_k\sum _{i=1}^m{x}_i^{j+k}=\sum _{i=1}^m{y}_i{x}_i^j,\text{for each }j=0,1,\dots ,n.$$

This is derived from the fact that the error is

$$E=\sum _{i=1}^m(y_i-P_n(x_i){)}^2.$$

The system of equations is produced to minimize $E$ by setting $0=\frac{\partial E}{\partial a_j}$ for each $j=0,1,\dots ,n.$

Continuous Least Squares Approximation

If we have a function $f\in C[a,b],$ we can use a similar method to approximate it with a polynomial of degree $n.$ This time, our $(n+1)$ normal equations are, for each $j=0,1,\dots ,n,$