Skip to main navigation Skip to search Skip to main content

Adaptive Clipping Strategy for Proximal Policy Optimization in Autonomous Vehicle Decision Making

  • Abhimanyu Singh
  • , D. Saxena
  • , Rajesh Kumar
  • , Naveen Gehlot*
  • *Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

Abstract

One of the main tasks of autonomous vehicles driving on highways is lane keeping based on intelligent decisions. However, this becomes challenging in constrained and complex traffic environments due to unstable policies, limited reward structures, and the need for real-time decision-making. To address these, this work presents a reinforcement learning-based approach using Proximal Policy Optimization (PPO) for highway lane keeping and obstacle avoidance. An accurate simulation environment is built using Unity and ML-Agents. PPO employs a clipping strategy designed to prevent large policy updates. These large policy updates can cause unstable training; however, the impact of different clipping strategies on training stability is significantly affected. To address this, this work presents a new clipping strategy, which is named Adaptive Clipping. To understand the effects of clipping parameter strategies on policy learning within PPO's clipped surrogate loss, four types of clipping strategies, constant, linearly decaying, exponentially decaying, and stepwise decaying, are compared with the proposed one. Each strategy is evaluated under the same environmental configuration to have a comprehensive comparison. The results demonstrate that the adaptive clipping strategy achieves highly accurate performance with a final average reward of 301.74 and a mean test success rate of 90.66%. The constant, stepwise, and exponentially decaying strategy presents comparable but slightly lower performance, while the linearly decaying strategy represents slightly worse results with the test success rate of 76.66%. Eventually, this work contributes a complete framework from Unity simulation to policy learning using discrete action control, vehicle-mounted sensors, and a robust reward function.

Original languageEnglish
JournalIEEE Transactions on Industry Applications
DOIs
Publication statusAccepted/In press - 2026

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Industrial and Manufacturing Engineering
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Adaptive Clipping Strategy for Proximal Policy Optimization in Autonomous Vehicle Decision Making'. Together they form a unique fingerprint.

Cite this