Optimization, Step 2

We’ll get really close to finding out why functions have minimums or maximums in this section. In order to do that, we’re going to need a few more tools. We’ll start with functions from \mathbb R \rightarrow \mathbb R, which are one dimensional in nature. Why are they one dimensional? The superscript on the first \mathbb R is implicitly a 1, i.e. \mathbb R^1 \rightarrow \mathbb R^1 ┬áin that we’re looking at functions of one independent variable. Soon we’ll generalize the results to n dimensions, and eventually to infinitely many dimensions, but we have to start with one dimension.
Continue reading “Optimization, Step 2”

Optimization, Step 1

Some preliminaries.

Engineers often take for granted that functions have minimums and maximums. In general, the underlying goals of machine learning and optimization are to find the minima and maxima of functions. As things get more complicated and more complex ideas as thrown into the mix of real-world problems, it’s not always clear that a goal of finding these maxima or minima is even attainable. Luckily there is a solid framework from which mathematicians stand in order to know that algorithms are going to achieve their goals, i.e. solutions will be found.
Continue reading “Optimization, Step 1”