Numerical analysis is the analysis and development of numerical methods for the solution of various mathematical problems, most particularly, the solution of (systems of) equations (both linear and nonlinear), approximation, optimization, the solution of ordinary and partial differential equations, and the computation of integrals and derivatives. It is different from other areas related to computation in that the quantities involved are real-valued, and so cannot be represented exactly in a finite amount of memory. A corollary of this is that almost all operations done in a computer involving allegedly real-value quantities produce incorrect results. The issue is the size of the errors produced. If these are small, then the results are accurate (but usually not exact), and can be used for practical purposes.
Numerical methods and their analysis revolve around several aspects:
Calculus, basic algebra.
There are two main approaches to the determination of errors in numerical computation.
Forward error analysis, and
Backward error analysis.
With forward error analysis, we begin with the initial data and follow the computation, and the error each step incurs. The error at the end of the computation gives the error in the result. The problem with this approach is that we can usually only work with bounds on the error, and operating on bounds commonly results in excessively large bounds on the errors in the results.
J.H. Wilkinson began the other approach which is to show that the computed results of a certain computation are the exact results of another problem of the same type with slightly perturbed data. This is backward error analysis, which he used with great success in studying Gaussian elimination/LU factorization for the solution of linear systems of equations.
Backward error analysis cannot be done for all numerical computations, and depends on the exact problem class, and how the data for a problem is represented. Nevertheless, it has been extremely successful, and has found new application outside of numerical linear algebra in areas such as symplectic methods for Hamiltonian ODE's.
The development of numerical methods is another sub-area where there are a great many "tricks" and ideas are used. There are a few general principles that can be used for developing numerical methods:
There are also a number of non-traditional tasks which use numerical methods (such as Google's page rank algorithm).
Subareas of numerical analysis:
The techniques or "tricks" used in numerical methods and numerical analysis are very broad, and range from the trivial to some very deep ideas.