Optimal Charging and Discharging Scheduling of Electric Vehicles using Reinforcement Learning for Efficient Operation

Abstract

The rapid growth of electric vehicles (EVs) has increased the demand for efficient charging and discharging strategies. This paper addresses EV scheduling in small cities to enhance charging efficiency and reduce user costs. We propose a hybrid framework that combines Deep Deterministic Policy Gradient (DDPG) with Hindsight Experience Replay (HER) and employs a Genetic Algorithm (GA) for hyperparameter optimization. The approach simultaneously targets three objectives: maximizing aggregator profit, minimizing EV charging cost, and maximizing the proportion of EVs departing with the target State of Charge (t-SOC). Simulations are performed under three scenarios with different aggregator capacities and EV counts: Scenario 1 (50 kW, 100 EVs), Scenario 2 (500 kW, 750 EVs), and Scenario 3 (50 kW, 150 EVs). Results indicate that the optimized Reinforcement Learning (RL) model outperforms conventional and other RL approaches like First Come First Serve (FCFS), Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC). Aggregator profits increase from $148–$503 (FCFS), $158–$530 (PPO), $162–$545 (SAC) to $181–$571 (optimized RL); EV charging costs decrease from $0.17–$0.25/kWh (FCFS), $0.12–$0.15/kWh (PPO), $0.11–$0.14/kWh (SAC) to $0.08–$0.09/kWh (optimized RL); and the percentage of EVs achieving t-SOC rises from 55–71% (FCFS), 76–85% (PPO), 78–87% (SAC) to 87–94% (optimized RL). These findings demonstrate that the proposed DDPG + HER + GA framework effectively balances multiple objectives while adhering to operational constraints, providing a robust solution for real-time EV charge-discharge scheduling.