Jing Zhang

Doctoral Dissertation Announcement

---

Candidate: Jing Zhang

Degree of: Doctor of Philosophy

Department: Computer Science

Title: Efficient Reinforcement Learning in Multiple-Agent Systems and Its Application in Cognitive Radio Networks

Committee:
Dr. Dionysios Kountainis, Chair
Dr. Ala Al-Fuqaha
Dr. Liang Dong
Dr. Leszek Lilien

Date: Friday, February 10, 2012 10:00 a.m. to Noon
D-210 Parkview Campus

Abstract:
The objective of reinforcement learning in multiple-agent systems is to find an efficient learning method for the agents to behave optimally. Finding Nash-equilibrium has become the common learning target for the optimality in reinforcement learning literature. However, finding Nash-equilibrium is a PPAD(Polynomial Parity Arguments on Directed graphs)-complete problem. The convention methods can find the Nash-equilibrium for some types of Markov games.
This dissertation proposes a new reinforcement learning algorithm to improve the search efficiency and effectiveness for multiple-agent systems. This algorithm is based on the definition of Nash-equilibrium. The algorithm utilizes the greedy and rational features of the agents. When the agents adjust their behavior strategies following certain rules based on the feedback, their behavior strategies display special patterns. The special patterns are tightly related to the Nash-equilibrium. The agents can find their Nash-equilibrium strategies according to the patterns’ properties even though each of the agents doesn’t have information about the other agents.
The new reinforcement learning algorithm can be applied in many areas as long as the target problem can be mapped to a Markov game. How to share dynamical spectrum resource efficiently is an important problem in cognitive radio networks. The dissertation shows a method that maps the resource sharing problem to a Markov game and solves the problem by applying the new learning algorithm.
There are several contributions of this research. First, the proposed reinforcement learning algorithm for multiple-agent systems doesn’t require the agents to know the information about other agents. Second, our learning algorithm is more efficient than other similar learning algorithms. Third, the learning algorithm also effectively finds a Nash-equilibrium. Fourth, the learning algorithm attempts to impact the learning problem in multiple-state Markov games. The algorithm is expected to be extended to the scalable Markov games. Last but not least, this reinforcement learning algorithm can be applied in many areas. We have applied the learning algorithm to find solution to the spectrum sharing problem in a cognitive radio network model is demonstrated in the dissertation.