Evaluating Machine Learning-Based Reinforcement Algorithms for Inverted Pendulum Stabilization

Abstract

This work is based on classical and modern research on reinforcement learning, utilizing the CartPole-v0 environment for Q-learning, Hill Climbing, the REINFORCE algorithm, various versions of the Deep-Q-Network algorithm, and policy gradient approaches such as PPO and A2C. All the above models have been implemented and compared in terms of convergence speed, stability, efficiency, and robustness in noisy environments, as well as their generality in modified environments. In the experiments, the optimization of the hyperparameters and other algorithmic modifications, such as the Double DQN, Dueling DQN, and Reward Shaping, have also been attempted. When compared with Classical Control approaches like the Linear Quadratic Regulator, the flexibility, stability, and efficiency of Deep RL approaches prove to be far better. The use of reward curves, policy heatmaps, and trajectory plots has proved beneficial as a way of ensuring the algorithm's interpretability. The results demonstrated that the reinforcement learning algorithms PPO and Dueling DQN are the most efficient methods with good convergence rates and stability, while providing insights on computational efficiency. This research created a hybrid framework that combines the use of Hill Climbing and reinforcement learning as a way of improving stability while ensuring better efficiency through the introduction of stability reward shaping techniques, which proved to be better than the use of classical reinforcement learning and control approaches.