UNIVERSITY PARK, Pa. — Traffic signals serve to regulate the worst bottlenecks in highly populated areas but are not always very effective. Researchers at Penn State are hoping to use deep reinforcement learning to improve traffic signal efficiency in urban areas, thanks to a one-year, $22,443 Penn State Institute for CyberScience Seed Grant.
Urban traffic congestion currently costs the U.S. economy $160 billion in lost productivity and causes 3.1 billion gallons of wasted fuel and 56 billion pounds of harmful CO2 emissions, according to the 2015 Urban Mobility Scorecard. Vikash Gayah, associate professor of civil engineering, and Zhenhui "Jessie" Li, associate professor of information sciences and technology, aim to tackle this issue by first identifying machine learning algorithms that will provide results consistent with traditional (theoretical) solutions for simple scenerios, and then building upon those algorithms by introducing complexities that cannot be readily addressed through traditional means.
“Typically, we would go out and do traffic counts for an hour at certain peak times of day and that would determine signal timings for the next year, but not every day looks like that hour, and so we get inefficiency,” Gayah said. “As we develop vehicles that can communicate with infrastructure, such as connected vehicles, and detection technologies that can see where the vehicles are, we can use artificial intelligence to make decisions about signal timing in real time in order to develop a much more efficient traffic signal.”
The team will use a deep reinforcement learning framework to construct a deep Q-network in order to develop a deep learning model. Real-world traffic data will be obtained from traffic cameras in various cities in China, including vehicle arrival times and speeds to numerous intersections in order to test the proposed methods against actual travel patterns instead of synthetic patterns that do not reflect real-time variability.
The goal of their study will be to show that a reinforcement learning-based approach can significantly outperform traditional adaptive traffic signal control methods.
Through this research, the team hopes to address past challenges, including how to generalize the learning methods to deal with diverse road structures; how to coordinate thousands of signals; whether there is a need to build a tiered control and how to do so; and how to create a simulator that can provide data samples in a more efficient and realistic way.
Past attempts to employ deep learning to traffic signal patterns have failed, Gayah said, because researchers have not thought carefully or thoughtfully about how to design it. In deep learning, the algorithm is based on a reward system. The traffic signal is given a situation and must then decide what to do. Once it does this, it learns if it was a good decision or a bad decision. If it makes a good decision, it is rewarded and thus tries to make increasingly better decisions in order to reap the rewards. The key is to give the signal all the right tools to do this successfully.
“If it's not thoughtfully designed, then it doesn't ask itself the right questions,” Gayah said. “And you can have a very big disconnect between the performance, and what you think you're going to get. It might also take the signal a long time to learn what to do, which means poor signal performance in the interim.”
Gayah described the strategies as black boxes. The researcher puts data into the black box, tells it what to do, and it does it.
“We're trying to take the black box off and really understand what's happening inside, so it’s more theoretically informed,” Gayah said.
The team hopes this one-year study will provide preliminary data that can then be used for additional funding to advance the research.
“We're moving towards computer-controlled everything,” Gayah said. “There are a lot of smart city initiatives in the United States and part of that is to develop the infrastructure to be as intelligent as possible. Using deep reinforcement learning will help us reach that goal.”
About the Penn State ICS Seed Grant program
The Penn State Institute for CyberScience created the ICS Seed Grant Program as a way to advance computation-enabled and data-enabled research by Penn State faculty. ICS Seed Grants are intended to support interdisciplinary research groups as they prepare to pursue major awards from external funding agencies by providing an initial investment that lays the groundwork for the larger proposal. Grants are also available to support the organization of a national or international research conference, held at Penn State, that encourages computation-enabled and data-enabled interdisciplinary research.