Adapting to game trees in zero-sum imperfect information games

The paper titled "Adapting to game trees in zero-sum imperfect information games" discusses the problem of learning $\epsilon$-optimal strategies in zero-sum imperfect information games (IIGs) through self-play with trajectory feedback. In IIGs, each player only partially observes the current game state, which makes it challenging to determine the optimal strategies. The authors provide a problem-independent lower bound on the required number of realizations to learn these strategies with high probability. The lower bound is given as $\widetilde{\mathcal{O}}(H(A_{\mathcal{X}}+B_{\mathcal{Y}})/\epsilon^2)$, where $H$ represents the length of the game and $A_{\mathcal{X}}$ and $B_{\mathcal{Y}}$ are the total number of actions for the two players. To address this problem, the authors propose two Follow the Regularized Leader (FTRL) algorithms: Balanced FTRL and Adaptive FTRL. The Balanced FTRL algorithm matches the lower bound but requires prior knowledge of the information set structure to define regularization. On the other hand, Adaptive FTRL progressively adapts regularization to observations without requiring prior knowledge of the information set structure. However, Adaptive FTRL needs $\widetilde{\mathcal{O}}(H^2(A_{\mathcal{X}}+B_{\mathcal{Y}})/\epsilon^2)$ realizations. The paper also provides a table summarizing different algorithms' sample complexity for episodic, finite, two-player, zero-sum IIGs. The sample complexities are expressed in terms of $H$, $A_{\mathcal{X}}$, $B_{\mathcal{Y}}$, and $\epsilon$. It is worth noting that some algorithms mentioned in Table 1 do not require prior knowledge of the information set structure. Overall, this paper contributes to our understanding of learning $\epsilon$-optimal strategies in zero-sum IIGs and provides algorithms with different trade-offs in terms of sample complexity and prior knowledge requirements.

Adapting to Game Trees in Zero-Sum Imperfect Information Games

Imperfect information games (IIGs) are a type of game where each player only partially observes the current game state. This makes it difficult for players to determine the optimal strategies, as they must take into account their incomplete knowledge of the game. In this paper, "Adapting to Game Trees in Zero-Sum Imperfect Information Games", the authors discuss how to learn $\epsilon$-optimal strategies in zero-sum IIGs through self-play with trajectory feedback.

Problem Definition and Lower Bound

The authors provide a problem-independent lower bound on the required number of realizations to learn these strategies with high probability. The lower bound is given as $\widetilde{\mathcal{O}}(H(A_{\mathcal{X}}+B_{\mathcal{Y}})/\epsilon^2)$, where $H$ represents the length of the game and $A_{\mathcal{X}}$ and $B_{\mathcal{Y}}$ are the total number of actions for two players X and Y respectively.

Proposed Algorithms

To address this problem, two Follow The Regularized Leader (FTRL) algorithms were proposed: Balanced FTRL and Adaptive FTRL. Balanced FTRL matches the lower bound but requires prior knowledge of information set structure to define regularization. On the other hand, Adaptive FTRL progressively adapts regularization without requiring prior knowledge but needs $\widetilde{\mathcal{O}}(H^2(A_{\mathcal{X}}+B_{\mathcal{Y}})/\epsilon^2)$ realizations instead.

Table 1 Summary

The paper also provides a table summarizing different algorithms' sample complexity for episodic, finite, two-player, zero-sum IIGs expressed in terms of $H$, $A_{\mathcal{X}}, B_ { \ mathcal {Y} }$, and $\epsilon$. It is worth noting that some algorithms mentioned do not require prior knowledge about information set structure.

Conclusion

Overall, this paper contributes to our understanding of learning $\epsilon$-optimal strategies in zero sum IIGs by providing algorithms with different tradeoffs between sample complexity and prior knowledge requirements.