Improving Dynamic Task Performance with Distributional Reinforcement Learning: A RealTime Fruit Slicing Control Case Study

Traditional scalar-based reinforcement learning (RL) methods like Advantage Actor-Critic (A2C) estimate only the expected return, which overlooks the full return distribution. This limitation often leads to suboptimal and unstable policies, especially in high-speed, continuous action spaces. To address these limitations, this paper proposes a Distributional A2C (DA2C) algorithm inspired by the principles of Distributional Soft Actor-Critic (DSAC). Instead of relying on scalar value estimation, proposed DA2C models return distributions as Gaussian, allowing the algorithm to learn the mean and variance of returns. This representation allows the agent to make more accurate value estimates, reducing the risk of over/under estimation. Consequently, policy updates become more stable, precise, and resilient to environmental variability. The proposed method is evaluated using a real-time simulated fruit-cutting task, which is a high-speed control problem that requires quick and precise decision-making. Simulation results show that the proposed DA2C outperforms the traditional A2C baseline across key metrics. Notably, it achieves higher slicing precision, faster response, and enhanced policy robustness, showcasing the benefits of incorporating distributional value estimation in complex, time-sensitive RL applications.