Distributed Deep Reinforcement Learning for Renewable Energy
Nowadays, microgrids (MG) have attracted much attention, as a key technology of the Internet of Energy (IoE). A great deal of research have shown that the hierarchical microgrid is a more novel structure of IoE. Although the hierarchical microgrid model solves the problem of weak power scheduling capability across microgrids, it suffers from severe communications uncertainty, which can lead to communication delay and fluctuation. To obtain the accurate result of the renewable energy accommodation assessment capacity, a hierarchical microgrid model considering communication uncertainty is investigated. The solution to solve the problem of the assessment renewable energy accommodation capacity for hierarchical MG is a hybrid control based on distribution deep reinforcement learning. The temporal difference (TD) generation adversarial network (TD-GAN) is proposed as a value-based method. Compared with the policy-based method, it can better solve the distributed problem in hybrid control with a generation adversarial network (GAN). Moreover, the challenge that the method cannot handle a continuous action space is solved by using a normalized advantage function (NAF). A method similar to the TD error method is employed to train the GAN network. Simulation results using real power grid data demonstrate the effectiveness and accuracy of the proposed method.
Zhu Han received the B.S. degree in electronic engineering from Tsinghua University, in 1997, and the M.S. and Ph.D. degrees in electrical and computer engineering from the University of Maryland, College Park, in 1999 and 2003, respectively. From 2000 to 2002, he was an R&D Engineer of JDSU, Germantown, Maryland. From 2003 to 2006, he was a Research Associate at the University of Maryland. From 2006 to 2008, he was an assistant professor at Boise State University, Idaho. Currently, he is a John and Rebecca Moores Professor in the Electrical and Computer Engineering Department as well as in the Computer Science Department at the University of Houston, Texas.
His research interests include wireless resource allocation and management, wireless communications and networking, game theory, big data analysis, security, and smart grid. Dr. Han received an NSF Career Award in 2010, the Fred W. Ellersick Prize of the IEEE Communication Society in 2011, the EURASIP Best Paper Award for the Journal on Advances in Signal Processing in 2015, IEEE Leonard G. Abraham Prize in the field of Communications Systems (best paper award in IEEE JSAC) in 2016, and several best paper awards in IEEE conferences. Currently, Dr. Han has been an IEEE fellow since 2014, an AAAS fellow since 2019, and an ACM distinguished member since 2019. Dr. Han is 1% highly cited researchers according to Web of Science since 2017. Dr. Han is also the winner of the 2021 IEEE Kiyo Tomiyasu Award, for outstanding early to mid-career contributions to technologies holding the promise of innovative applications, with the following citation: “for contributions to game theory and distributed management of autonomous communication networks.”