# A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections

**Authors:** Mengyu Guo, Pin Wang, Ching-Yao Chan, Sid Askary  
**arXiv:** [1905.07698](https://arxiv.org/abs/1905.07698)  
**Date:** 2019-05-19  
**Summary page:** https://www.summarizepaper.com/en/paper/1905.07698v1/

## Key points

- Traffic signal control at urban intersections can lead to traffic bottlenecks, congestion, delays, and environmental issues.
- Recent advancements in machine learning techniques, such as reinforcement learning (RL), offer a promising solution.
- The proposed approach uses RL and neural networks as Q-function approximators (Q-networks) for traffic signal control.
- Real-time traffic information, including vehicle position, direction, and speed, is used to define the state space.
- The action space includes various traffic signal phases that consider the spatial-temporal characteristics of urban traffic.
- Simulation experiments are conducted using SUMO - an open source traffic simulator - to evaluate the proposed approach.
- Results show that the RL approach outperforms benchmarking methods in terms of reducing queue length and wait time.
- This research highlights the potential of using advanced machine learning techniques for intelligent traffic signal control at urban intersections.

## Summary

In urban areas, traffic signal control at intersections can lead to traffic bottlenecks, congestion, delays and environmental issues. To address this problem, recent advancements in machine learning techniques such as reinforcement learning (RL) offer a promising solution. This paper proposes an RL approach for traffic signal control at urban intersections using neural networks as Q-function approximators (referred to as Q-networks). The state space is defined based on real-time traffic information such as vehicle position, direction and speed while the action space includes various traffic signal phases crucial for generating a reasonable and realistic control mechanism considering the spatial-temporal characteristics of urban traffic. To evaluate the proposed approach, simulation experiments are conducted using SUMO - an open source traffic simulator that allows them to create realistic urban intersection settings with different traffic patterns like major/minor road traffic, through/left-turn lane traffic and varying demand traffic. Simulation results demonstrate the convergence and generalization performance of the RL approach proposed by the authors compared to several benchmarking methods in terms of reducing queue length and wait time. This research highlights the potential of using advanced machine learning techniques like reinforcement learning for intelligent traffic signal control at urban intersections which offers an effective means of managing congestion and improving overall transportation efficiency.