A Study of Gradient-Based Algorithms
Gradient-based algorithms are popular when solving unconstrained optimization problems. By exploiting knowledge of the gradient of the objective function to optimize, each iteration of a gradient-based algorithm aims at approaching the minimizer of said function. In the age of web-scale prediction problems, many venerable algorithms may encounter difficulties. However, as the data sets grow larger,