The Wikipedia article for compact spaces is large, but extremely informative. I want to distill the important points of what it means to be compact in the context of optimization.
The motivation for something to be compact is a generalization of being closed and bounded. Indeed, any subset that is both closed and bounded is compact. Take this as a definition for now, and we’ll expand on it and re-adjust our thinking. Recall that is both closed and open, but it is definitely not bounded, as it has no upper or lower bound. Without loss of generality, we will only consider closed intervals, i.e. . Being closed means that contains its endpoints, but in general it means that a sequence with terms in has a limit that also stays within . We would like it if compactness would mean just that – that sequences with terms in have their limit point in . Is the ‘definition’ of compactness we used at the beginning of this paragraph consistent with this logic? Yes and no, and we’ll delve into that now.
Continue reading “Optimization, Step 4: Compactness”