Another part of our series called “Reinforcement Learning in practice”! AI playing games.
Today we leave text-based games and we do first steps in graphical environment! We will start with “beginner classic” - CartPole!
- Frozen Lake
- Lunar Lander
All the details explained HERE
Game looks like that:
Again - it is extremely simple - we have to control the cart to prevent the pole attached from falling over. We have only 2 actions available - we can move our vehicle left or right - and for each step we get 1 point.
In this excercise we use version 0 of environment which is limited to 200 steps.
Here we not only abandon text games but also tabular methods. From now on we will be using Neural Networks as optimal function approximators.
In fact in this case we were testing multiple methods and definitely the best is a combination of Deep Q Network (Deepmind’s DQN) with some improvements - Double DQN and Dueling DQN.
Thanks to that we achieved really great results. As always we used Pytorch.
Rewards and steps in this case are pretty much the same:
Info: X axis is number of episodes, Y axis is score (rewards received from environment per episode).
As we can see our agent very quickly figured out what’s going on in the environment.
It was able to reach maximum number of steps at 14th episode!
In fact we were able to solve the game already after 109 episodes! (by solving the game it is considered to have at least 195 points as an average reward for last 100 episodes).
This is very good result!
Below - agent’s performance captured on video:
Like it? Want to do similar things?
Have a look at our course! We teach everything from scratch!