Feb 25, 2021
What’s with all of the buzzwords? — My project in non-AI terms.
When I started my new position as a PhD. student at Intelligent Games and Game Intelligence (IGGI) I was really hyped to talk about my project. PhD in video games! Who wouldn’t be excited? But whenever one of my colleagues asked me about my exact project title, their face went blank.
Even if you are a machine learning specialist this seems daunting. Everybody expects something like “New ways to play VR games” or “Making a bot for Starcraft 2”, not something that looks like a bunch of buzzwords thrown into one sentence. So here it goes, step by step instructions for understanding my project. It’s much cooler than a new VR game, or a bot for Starcraft.
Let’s split it into four parts.
Part 1 — Local Forward Model Learning
If I ask you “Can you predict the future?”. The answer is most likely “No!”. But I can ask the same question in a different way. Let’s imagine that you are at your desk, reading a blog. While reading, you suddenly notice that you are really thirsty, and you would like some tea. If I ask you to describe what your surroundings will look like in 10 minutes, could you tell me?
It will probably look the same. You will be wearing the same clothes, sitting in the same chair, at the same desk, reading the same blog post, just with a cup of tea beside you. There you go! You predicted the future. That’s what a Forward Model does. You give it some information about the environment and what you plan on doing, and it tells you what will happen next.
The word “local” in “local forward model” means that you are only interested in your surroundings. Usually what happens in another city doesn’t affect your tea. That translates into the game world as well. What happens on the other side of the game world doesn’t really affect you that much.
And finally learning. Learning means that the Forward Model predicts the future by trial and error. Just like a human. How did you know that the cup of tea is going to be on your desk? Because you have done the same actions many times. And you learned that the outcome of making tea is tea in your cup.
Part 2 — Sample Efficient
Let’s move on to the next part. Artificial Intelligence (AI) is all about efficiency. ‘Sample efficient’ means it doesn’t require that many trial and errors. If we look at the samples needed to learn a game, it is immense. The computational power required for the creation of an AI agent is out of the scope for most research groups.
For an AI to learn Starcraft, it required 200 years worth of real-time play. 200 years of trials and errors, per agent, and they trained a bunch of them. That amount of training time costs ~€12 million. Only a select few can afford such an experiment.
Part 3 — Sequential Decision Making
Sequential decision making. Just another way of saying we’ll cross that bridge when we come to it. You don’t need to know everything in advance. When you think about making a cup of tea what are the steps that you need to do to make it?
You probably have an idea similar to this:
But actually, you missed some steps:
If someone showed me the second example as a way to make tea I would think that they are crazy. You still have to do all 22 steps, but you think about them only when you get to them.
The second picture shows how we teach a standard AI. Let’s not do that, let’s teach it the simple one first. And when it wants to add milk, then we can show it how to open and close the fridge.
Part 4 — Open World 3D Games
Open world 3D games. Not a buzzword like others, so why is it here? This is the part that everybody wants to hear about when they think of a PhD. in Game AI. Which games are you working on?? Well every game, that’s what the last part of the project title is about. People can play any game, why couldn’t an AI. If we try to make them think the same way that we do, then they should be able to play the same games.
So, there it is. My new project title. No buzzwords! I hope that my project doesn’t seem too daunting now.
If you are interested in more projects in the field of video games take a look at IGGI. If any of the topics there don’t make sense, feel free to send me a tweet and we can have a chat. I LOVE taking about these things!
See you around,
Marko
