Skip to search formSkip to main contentSkip to account menu

Zap Q-Learning

The Zap Q-learning algorithm introduced in this paper is an improvement of Watkins' original algorithm and recent competitors in several respects and suggests that the approach will lead to stable and efficient computation even for non-ideal parameterized settings.
View Paper (opens in a new tab)

Model-Free Primal-Dual Methods for Network Optimization with Application to Real-Time Optimal Power Flow

This paper examines the problem of real-time optimization of networked systems and develops online algorithms that steer the system towards the optimal trajectory without explicit knowledge of the system model, and leverages the online zero-order primal-dual projected-gradient method.
IEEE (opens in a new tab)

Explicit Mean-Square Error Bounds for Monte-Carlo and Linear Stochastic Approximation

It is shown that mean square error achieves the optimal rate of $O(1/n)$, subject to conditions on the step-size sequence, which is of great value in algorithm design.
arXiv (opens in a new tab)

Fastest Convergence for Q-learning

The Zap Q-learning algorithm introduced in this paper is an improvement of Watkins' original algorithm and recent competitors in several respects. It is a matrix-gain algorithm designed so that itsโ€ฆ
arXiv (opens in a new tab)

The ODE Method for Asymptotic Statistics in Stochastic Approximation and Reinforcement Learning

The main results now allow for parameter-dependent noise, as is often the case in applications to reinforcement learning.
arXiv (opens in a new tab)

Q-Learning With Uniformly Bounded Variance

It is shown that the asymptotic covariance of the tabular Q-learning algorithm with an optimized step-size sequence is a quadratic function of a factor that goes to infinity, as discount factor approaches 1; an essentially known result.
IEEE (opens in a new tab)

On Matrix Momentum Stochastic Approximation and Applications to Q-learning

It is shown that the parameter estimates obtained from the PolSA algorithm couple with those of the optimal variance SNR algorithm, at a rate of O(1/n^{2})$, and numerical results confirm the coupling of PolSA and SNR.
IEEE (opens in a new tab)

Zap Q-Learning With Nonlinear Function Approximation

This class of algorithms is generalized in this paper, and stability is established under very general conditions, and this general result can be applied to a wide range of algorithms found in reinforcement learning.
arXiv (opens in a new tab)

Stochastic Variance Reduced Primal Dual Algorithms for Empirical Composition Optimization

This work reforms the original minimization objective into an equivalent min-max objective, which brings out all the empirical averages that are originally inside the nonlinear loss functions, and develops a stochastic primal-dual algorithm, SVRPDA-I, which is shown to converge at a linear rate when the problem is strongly convex.
arXiv (opens in a new tab)

Revisiting the ODE Method for Recursive Algorithms: Fast Convergence Using Quasi Stochastic Approximation

A brief survey of recent research in machine learning that shows the power of algorithm design in continuous time, following by careful approximation to obtain a practical recursive algorithm.
Springer Nature (opens in a new tab)
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy (opens in a new tab), Terms of Service (opens in a new tab), and Dataset License (opens in a new tab)