Download
Abstract

Reinforcement learning (RL) is centered on how autonomous agents interact in an environment to maximize a pre-defined cumulative reward. Since the introduction of fairness in RL by Jabbari et al. in 2017, there’s been an increasing interest in exploring the theory and applications of fairness metrics in this domain. This literature review delves into the current state of research, comparing various approaches and highlighting potential future research directions.

In traditional RL, the focus is on maximizing the long-term discounted reward of an agent tailored to a specific objective, such as game points or stock market gains. However, this doesn’t account for considerations of group or individual fairness. Ignoring fairness can lead to negative consequences, both legally and in terms of user acceptance. For instance, if an RL algorithm promotes content from one demographic over another on a platform like YouTube, it can lead to reduced diversity and potentially drive away users.

The paper discusses various definitions of fairness, including:

  • Welfare Economics Definitions: Fairness is viewed in terms of social welfare functions, considering components like impartiality, equity, and efficiency.
  • Weighted Proportional Fairness: Used in interactive recommender systems, it looks at the allocation proportion of different groups.
  • Coefficient of Variation: Measures fairness in multi-agent systems by looking at the distribution of resources or rewards among agents.
  • Q-Value Based Definitions: Focuses on the expected utility of actions in a Markov Decision Process.
  • α-Fair Utility: Originating from computer networking, this definition allows for adjustable levels of fairness.

The paper also touches upon the application domains of fairness in RL, particularly in decision support systems and autonomous systems. Examples include interactive recommendation systems, human-robot team resource distribution, wireless network scheduling, and traffic light control.

In conclusion, while fairness in RL has gained traction, it remains an understudied field with diverse and sometimes conflicting definitions. The paper aims to provide a comprehensive review of the current state of fairness in RL and offers insights into potential future research directions.