Deep Reinforcement Learning for Energy Management in Hybrid Electric Vehicles with Softmax Double-Actor Regularized Critics

Enhancing fuel efficiency in hybrid electric vehicles (HEVs) requires energy management strategies (EMSs) that can operate effectively under nonlinear powertrain dynamics and uncertain, time-varying driving conditions. This paper proposes a deep reinforcement learning (DRL)- based EMS using the double actors regularized critics softmax deep deterministic policy gradient (DARC SD3) algorithm, which integrates Boltzmann-softmax value estimation, a dual-actor architecture, and critic regularization to improve learning stability and value-estimation accuracy. Simulation results show that the proposed DARC SD3 achieves faster convergence, improved state-of-charge (SOC) regulation, and reduced value estimation bias compared with DDPG, TD3, and baseline SD3. Under the FTP-75 driving cycle, the proposed EMS attains 94.6% of the dynamic programming (DP) benchmark fuel economy, while reducing engine transients and smoothing battery power flow. Further evaluation on an unseen composite driving cycle confirms that the trained policy maintains consistent fuel economy and SOC control, demonstrating strong generalization capability across diverse driving conditions.