How to Use OpenAI’s Gym tool Library and Universe platform to create AI Game Bot

Neon Racing Flash game environment on Universe platform

Let’s face it, ARTIFICIAL intelligence is everywhere. A battle is raging between Elon Musk and Mark Zuckerberg over the future of artificial intelligence. Some people demonize ARTIFICIAL intelligence; Some people idealize artificial intelligence, thinking that it can help human beings like a god. Whichever way you like to look at it, AI is not going away.

“> < p style =” max-width: 100%; clear: both; “Fearing the rise of killer robots is like worrying about overpopulation on Mars.” – Andrew Ng

If you’re interested in AI, and you want to move in that direction, and you want to play around with ai, making games is a good place to start, because games have always been the best test bed for AI. But before we get started, let’s talk a little bit about the history of game programming and how it has evolved from then to now.

The history of game programming

Game programmers used to make games by experimenting. Every decision made in the game is an educated guess based on if-then-else statements. This approach, seen in the earliest arcade games like Pong and PacMan, has long been the norm. But game developers are limited in what they can predict and need to consider extreme scenarios so that game machines don’t run around like headless chickens.

So game developers are trying to mimic the way humans play games, modeling human intelligence and incorporating it into gaming robots.

That’s what the DeepMind team did. They generalized and modeled intelligent behavior, and all Atari games that were pitched to them were solved this way. Their gaming robots use deep learning neural networks without any specific knowledge of the game. Game progression depends on the pixels you see on the screen and understanding the game’s controls. However, parts of DeepMind’s projects are still not open source, because its parent company, Google, is using DeepMind to compete in the marketplace.

Democratization of artificial Intelligence

Artificial intelligence has amazing power, and in order not to concentrate this power in the hands of a few people, Elon Musk founded OpenAI. The company seeks to democratize AI and make it accessible to everyone. Today we’re going to explore OpenAI’s Gym library and the recently released Universe platform based on this library.

OpenAI Gym provides a very simple interface for any dynamic environment to interact with and manage. OpenAI Universe is a platform where you can build robots and test them.

There are thousands of game environments. From classic Atari games, Minecraft, Grand Theft Auto to protein Fold simulations for cancer treatment. The ability to create a game robot that can run in any environment with just a few lines of Python code is fantastic and definitely not to be missed.

Project (1 hour)

We’re going to build an AI robot using Reinforcement Learning, which I’ll explain later. The robot can play against itself and beat Atari games, Neon Racers. Of course, if you want something else, you can. We will use OpenAI’s Gym tool library and Universe platform to create this game robot.

Step 1: Install

Make sure Python is installed, or Homebrew if not. You can download a Python-specific IDE editor, such as PyCharm or iPython Notebook. I prefer something simple, so I use the Sublime editor. Finally, install the Gym tool library, Universe platform and other necessary libraries with PIP.

// Install python brew install python3 // Install the required OpenAI libraries pip3 install gym pip3 install numpy Incremental brew install golang libjpeg-turbo pip install universeCopy the code

All parts of the Universe platform environment run as containers inside Docker. If Docer is not already installed, go here to install it and run it.

Step 2: Write the game robot

The game robot is written in Python. There are only two necessary libraries for this project: Gym tool library and Universe platform, so first enter it at the beginning.

import gym
import universe
Copy the code

The robot used my favorite game as a kid, Neon Racers, as a test environment. Other available environments or games are listed here, so you can choose your own.

The Universe platform enables multiple environments to run side by side, as many as needed. But for this project we only use one.

Env = gym.make(' flashgames.neonrace-v0 ') env.configure(remotes=1) # create a local docker containerCopy the code

Reinforcement Learning

Now we use reinforcement learning techniques to add game logic to the game robot. The technique works by looking at previous game states and rewards, such as pixels on the screen or game scores, and then figuring out what actions to take in that environment.

The purpose of this operation is to make the next observation better than the previous one. In our case, it’s to get as many points as possible. To get the highest score, the game medium — the game robot — selects and performs actions that are then imposed on the environment. The environment records new states and rewards based on whether the action is beneficial, such as whether the game is won.

Now, we can list observations from each initialized environment using the env.reset() method.

observation_n = env.reset()
Copy the code

The observations here are objects unique to the current environment. It represents observed information, such as raw pixel data on the screen or game state, score, etc.

The next step is to create a game medium with an infinite loop that continually takes certain actions based on observations. Let’s have a single action in the robot, which is to press the up arrow button over and over again, which is kind of silly, so let’s make it more complicated. The action here is defined by the event type, the KeyEvent, and the control key, the Up Arrow. This value is set to be true regardless of what the medium observes.

while True:
action_n = [[('KeyEvent', 'ArrowUp', True)] for ob in observation_n]
Copy the code

We then use the env.step() method to manipulate the action one step at a time. This enables very basic reinforcement learning techniques.

 observation_n, reward_n, done_n, info = env.step(action_n)
Copy the code

This walk method returns four variables:

  1. observation_n: Environmental observations
  2. reward_n: Whether the action is favorable or not, the result is +1 or -1
  3. done_n: Displays whether the game is over, and the result is Yes or No (Yes/No)
  4. info: Other information, such as performance, latency, etc. used to debug the process

To train the robot, all environments can run the action simultaneously. Use the env.render() method to start the robot.

env.render()
Copy the code

Now the game robot is ready to play in the environment. The complete code for the basic and advanced versions of the robot is available on my Github library site, here.

Step 3: Run the game robot

Now comes the fun part: make sure Docker is running and start the bot. Watch the robots in action, either beating the other cars or not. If you lose, fine-tune the robot to make it smarter!

python [gamebot.py](http://gamebot.py)
Copy the code

Crash and fire! Basic version #

Fiddling with artificial intelligence will unlock God mode! #100DaysOfCode