Another part of our series called “Reinforcement Learning in practice”! AI playing games.
Today we will try to make first steps into most interesting part of this area - learning directly from video output! We will go through one of the most popular Atari games - Pong!
- Frozen Lake
- Lunar Lander
All the details explained HERE
Game looks like that:
Of course rules are extremely simple. Our agent controls pad on the right hand side, has to look after the ball and score more points then his opponent on the left.
This time - as we want to learn directly from video output - we will use Convolutional Neural Networks (CNN), and additionally (like in previous cases) DQN with improvements - Double DQN and Dueling DQN. And as always - we keep using Pytorch.
Info: X axis is number of episodes, Y axis is score (rewards received from environment per episode).
Of course - as we are using CNN now - the learning period takes much longer then in previous, simple examples - although still - Pong is one of the games that can be solved relatively quickly.
We managed to solve it in around 75 minutes (using single NVidia GeForce GTX 1080) - exactly after 300 episodes!
Already around 120th episode our agent was winning single games and around 200th it was consistently dominating his opponent.
Average for last 100 episodes was 19.13 points so much better than human performance - which is around 9.3 points according to Deepmind’s paper.
Below - agent’s performance captured on video:
Like it? Want to do similar things?
Have a look at our course! We teach everything from scratch!