Numerical optimization is a large and important part of numerical analysis, with a great many practical applications.

We can focus on different types of problems, particularly, unconstrained and constrained (equality and inequality constrints). Constrained problems lead to the use of Lagrange multipliers. The objective function (the function to minimize or maximize) can be convex, smooth, or non-smooth (and convex or not as the case may be).

Non-convex problems are generally harder than convex ones, especially as there is often a problem of local minima (or local maxima).