Microsoft

Katyusha: The First Direct Acceleration of Stochastic Gradient Methods

Nesterov’s momentum trick is famously known for accelerating gradient descent, and has been proven useful in building fast iterative algorithms. However, in the stochastic setting, counterexamples ...
IEEE

Stochastic Gradient Descent for Nonconvex Learning Without Bounded Gradient Assumptions

Abstract: Stochastic gradient descent (SGD) is a popular and efficient method with wide applications in training deep neural nets and other nonconvex models. While the behavior of SGD is well ...
IEEE

Stochastic Gradient Descent on Riemannian Manifolds

Abstract: Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending ...
Hacker

Extending Stochastic Gradient Optimization with ADAM

Gradient descent is a method to minimize an objective function F(θ) It’s like a “fitness tracker” for your model — it tells you how good or bad your model’’ predictions are. Gradient descent isn’t a ...
Scientific Research Publishing

Friedman, J.H. (1999) Stochastic Gradient Boosting. Technical Report, Stanford University, Stanford.

ABSTRACT: Urban grid power forecasting is one of the important tasks of power system operators, which helps to analyze the development trend of the city. As the demand for electricity in various ...