A Cloud-based Sequential Optimization DDPG Strategy for Hybrid Vehicles Energy-efficiency Improvement

With the rapid development of vehicle-road-cloud integrated systems and artificial intelligence (AI) technologies, intelligent connected vehicles (ICVs) can be empowered with enhanced capabilities to address the inherent limitations of single-vehicle driving, such as restricted perception range and insufficient computational resources, making them an ideal platform for realizing future autonomous driving. However, there exists a notable research gap in applying AI to energy-efficient driving strategies for hybrid electric vehicles (HEVs), while simultaneously ensuring improved traffic efficiency. Therefore, this study presents a novel cloud-based sequential optimization deep deterministic policy gradient (SO-DDPG) framework addressing the energy-efficient and time-efficient control strategy for hybrid electric vehicles in urban environments. Firstly, a hierarchical “cloud decision-making, vehicle control” architecture is implemented by featuring an innovative integration of real-time energy management to achieve coordinated optimization of speed planning and power distribution. Secondly, a double-layer DDPG architecture is employed for control strategy learning. The first layer trains the energy management model in the cloud, taking vehicle motion states and powertrain system states as input variables to generate engine power control signals. The second layer integrates various information including traffic signal phases and preceding vehicle states to plan optimal velocity sequences. Finally, comprehensive validation experiments conducted on urban roads demonstrate the algorithm′s robust performance: Comparative analyses show that SO-DDPG achieves 6.05% higher energy efficiency than traditional hierarchy DDPG, and compared to green light optimal speed advisory (GLOSA) algorithm, it reduces energy consumption by 33.96% with only 1.90% increase in travel time.