Deep Reinforcement Learning for General Video Game AI

Ruben Rodriguez TORRADO; Philip BONTRAGER; Julian TOGELIUS; Jialin LIU; Diego PÉREZ-LIÉBANA

doi:10.1109/CIG.2018.8490422

Deep Reinforcement Learning for General Video Game AI

Ruben Rodriguez TORRADO, Philip BONTRAGER, Julian TOGELIUS, Jialin LIU, Diego PÉREZ-LIÉBANA

Research output: Book Chapters | Papers in Conference Proceedings › Conference paper (refereed) › Referred Conference Paper › peer-review

117 Citations (Scopus)

Abstract

The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.

Original language	English
Title of host publication	Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018
Editors	Cameron BROWNE
Publisher	IEEE Computer Society
Number of pages	8
ISBN (Electronic)	9781538643594
DOIs	https://doi.org/10.1109/CIG.2018.8490422
Publication status	Published - 2018
Externally published	Yes
Event	14th IEEE Conference on Computational Intelligence and Games, CIG 2018 - Maastricht, Netherlands Duration: 14 Aug 2018 → 17 Aug 2018

Publication series

Name	IEEE Conference on Computatonal Intelligence and Games, CIG
Volume	2018-August
ISSN (Print)	2325-4270
ISSN (Electronic)	2325-4289

Conference

Conference	14th IEEE Conference on Computational Intelligence and Games, CIG 2018
Country/Territory	Netherlands
City	Maastricht
Period	14/08/18 → 17/08/18

Bibliographical note

Publisher Copyright:
© 2018 IEEE.

Funding

This work was supported by the Ministry of Science and Technology of China (2017YFC0804003). (*) The first two authors contributed equally to this work.

Keywords

Advantage actor critic
Deep Q-learning
Deep reinforcement learning
General video game AI
OpenAI Gym
Video game description language

Access to Document

10.1109/CIG.2018.8490422

Cite this

TORRADO, R. R., BONTRAGER, P., TOGELIUS, J., LIU, J., & PÉREZ-LIÉBANA, D. (2018). Deep Reinforcement Learning for General Video Game AI. In C. BROWNE (Ed.), Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018 Article 8490422 (IEEE Conference on Computatonal Intelligence and Games, CIG; Vol. 2018-August). IEEE Computer Society. https://doi.org/10.1109/CIG.2018.8490422

@inproceedings{95432e7b74594829bde8e9fe7649d532,

title = "Deep Reinforcement Learning for General Video Game AI",

abstract = "The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.",

keywords = "Advantage actor critic, Deep Q-learning, Deep reinforcement learning, General video game AI, OpenAI Gym, Video game description language",

author = "TORRADO, \{Ruben Rodriguez\} and Philip BONTRAGER and Julian TOGELIUS and Jialin LIU and Diego P{\'E}REZ-LI{\'E}BANA",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 14th IEEE Conference on Computational Intelligence and Games, CIG 2018 ; Conference date: 14-08-2018 Through 17-08-2018",

year = "2018",

doi = "10.1109/CIG.2018.8490422",

language = "English",

series = "IEEE Conference on Computatonal Intelligence and Games, CIG",

publisher = "IEEE Computer Society",

editor = "Cameron BROWNE",

booktitle = "Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018",

address = "United States",

}

TORRADO, RR, BONTRAGER, P, TOGELIUS, J, LIU, J & PÉREZ-LIÉBANA, D 2018, Deep Reinforcement Learning for General Video Game AI. in C BROWNE (ed.), Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018., 8490422, IEEE Conference on Computatonal Intelligence and Games, CIG, vol. 2018-August, IEEE Computer Society, 14th IEEE Conference on Computational Intelligence and Games, CIG 2018, Maastricht, Netherlands, 14/08/18. https://doi.org/10.1109/CIG.2018.8490422

Deep Reinforcement Learning for General Video Game AI. / TORRADO, Ruben Rodriguez; BONTRAGER, Philip; TOGELIUS, Julian et al.
Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018. ed. / Cameron BROWNE. IEEE Computer Society, 2018. 8490422 (IEEE Conference on Computatonal Intelligence and Games, CIG; Vol. 2018-August).

Research output: Book Chapters | Papers in Conference Proceedings › Conference paper (refereed) › Referred Conference Paper › peer-review

TY - GEN

T1 - Deep Reinforcement Learning for General Video Game AI

AU - TORRADO, Ruben Rodriguez

AU - BONTRAGER, Philip

AU - TOGELIUS, Julian

AU - LIU, Jialin

AU - PÉREZ-LIÉBANA, Diego

PY - 2018

Y1 - 2018

N2 - The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.

AB - The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.

KW - Advantage actor critic

KW - Deep Q-learning

KW - Deep reinforcement learning

KW - General video game AI

KW - OpenAI Gym

KW - Video game description language

UR - https://www.scopus.com/pages/publications/85056836621

U2 - 10.1109/CIG.2018.8490422

DO - 10.1109/CIG.2018.8490422

M3 - Conference paper (refereed)

AN - SCOPUS:85056836621

T3 - IEEE Conference on Computatonal Intelligence and Games, CIG

BT - Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018

A2 - BROWNE, Cameron

PB - IEEE Computer Society

T2 - 14th IEEE Conference on Computational Intelligence and Games, CIG 2018

Y2 - 14 August 2018 through 17 August 2018

ER -

TORRADO RR, BONTRAGER P, TOGELIUS J, LIU J, PÉREZ-LIÉBANA D. Deep Reinforcement Learning for General Video Game AI. In BROWNE C, editor, Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018. IEEE Computer Society. 2018. 8490422. (IEEE Conference on Computatonal Intelligence and Games, CIG). Epub 2018 Oct 14. doi: 10.1109/CIG.2018.8490422

Deep Reinforcement Learning for General Video Game AI

Abstract

Publication series

Conference

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this