Usage of Machine Learning in games

Mert

Member
Hi.

I was wondering if you guys use machine learning/AI in your games. I'm currently working on Firebase Machine Learning extensions for GMS2, and I'm thinking about couple of things.

In which situations you'd use them?
Thanks
 

Yal

šŸ§ *penguin noises*
GMC Elder
This feels more like a gimmick than a mature solution, but it might just be me being biased against using things that you don't know how they work. Not sure if GML has enough computational power to make the necessary number-crunching viable, either.

Machine learning feels more useful for an offline solution (run tests during development, compile useful data for releases), if you're going to give the player a blank-slate AI that walks into walls you're not going to give off a good first impression. Once you deploy the game, though, you could have the machine-learning learn from real player data, and if you just can find a way to collect it (preferably GDPR-compliant) you could use it to improve the AI in later patches.
 

FrostyCat

Member
This feels more like a gimmick than a mature solution, but it might just be me being biased against using things that you don't know how they work. Not sure if GML has enough computational power to make the necessary number-crunching viable, either.

Machine learning feels more useful for an offline solution (run tests during development, compile useful data for releases), if you're going to give the player a blank-slate AI that walks into walls you're not going to give off a good first impression. Once you deploy the game, though, you could have the machine-learning learn from real player data, and if you just can find a way to collect it (preferably GDPR-compliant) you could use it to improve the AI in later patches.
GML won't be the one doing the computations, it's a third-party library. Mert's point here is offloading the AI to a pre-compiled solution in a faster language.

Also, even after you train the AI (which I also agree is best done in advance), you still need to load and apply that data in a way the game engine can use. At the very least, that part of the code still needs a GML-bound interface shipping with the final product.

While I think most of the machine learning attempts in action-heavy games are gimmicky, there is great potential for this library in more turn-based, strategy-heavy genres that GM has historically struggled with. As an example, I have a pure-GML implementation of MCTS (which some sources consider a part of machine learning), and though its performance has room for improvement, it could do what no other GM strategy project before it could do. My GMC Jam 31 entry contains zero game-specific AI code, only the rules of the game, the unaltered generic MCTS engine and some preset play strength parameters. Yet most reviewers are unable to beat it even at its default strength.
 
I think ML could create a new paradigm of gaming: But it's going to take a while for the technology to get adapted by game developers, but some smarties will make the breakthrough: Kind of like John Romero's Wolfenstein/Doom: Once the games' been made, we've been seeing Doom clones ever since.

So the obvious application is board games. We've already seen this with AlphaZero (Go, Chess, etc) who now dominate those professional scene's: (Although little * beside Chess as there's a bit of a question around unfair computational equivalence of the machines used).

So any board game: Hex, etc, could (and probably should) take a ML approach to it's AI.

For 'out of the box' new game paradigm's, I always suspected gesture recognition, hand-writing/character recognition, etc, which ML can help a lot with: Touch screen, mouse drawing, or even camera 'hand' recognition.

This feels more like a gimmick than a mature solution, but it might just be me being biased against using things that you don't know how they work.
To compare with something you'd be a bit more familiar with, I always felt they(or neural networks at least) were somewhat analogous to a PID Controller. You're essentially feeding an input into a model and tweaking it to produce an output. Only with the NN model, the more data you feed in, the more 'tuned' (or corrupted!) the model becomes.

Not a 1-to-1 example: But it works for me from a 'high level' view. And just like how PID's can be used just about anything, NN's can (and are being) used for just about everything people can think of.

Not sure if GML has enough computational power to make the necessary number-crunching viable, either.
In theory, once the model is trained it should be fast. But getting the input in GML might be difficult. When Deepminds taught their AI to play the old Atari classics: Breakout, Pacman, etc. The inputs were just the pixels on the screen. GML is *slow* at reading that kind of data: So this kind of pixels-in-> controller out approach would not work in GM!
 

Mert

Member
Thanks for the comments, yes the hardwork is offloaded to the big daddy Google servers. There are also On-Device models, which work on device and the work is still handled by the app, but in a faster way. I'm still at learning/creating phase, everyday I learn something new.

As of now, there are premade tools for different tools: I'm thinking about image labeling for an online Drawing App Game I'm thinking to work on. This is to ensure that trolls don't throw the games/rounds
  • Players join an online lobby.
  • User A is asked to draw a pencil.
  • User A draws a book instead.
  • AI detecs this and warns the user to draw a pencil, instead of book.
 

Nux

Member
I'm thinking about image labeling for an online Drawing App Game I'm thinking to work on
I think that's a good idea. Neural networks are great for classification problems.

Other than that, I can't think of any other place a machine learning algorithm would be useful in video games. I'd like to think that, in most cases, steering behaviours and locamotion solvers get the job done better than reinforcement learning ever could. Usually because, like Yal said, your agents will start out being very extremely dumb if you use machine learning or genetic algorithms.

I think the only time I could see them being useful is if you were going for something like the figure fighters from Super Smash Brothers.
 

NeutronCat

Member
It often takes hours or days to get decent result and GML will be even slower, but it doesn't really matter since you can use trained dataset in your released games.
It would be faster to train with GPU or on the cloud, but that's not easy.

I think machine learning has its potential in video games, and some other game engines already have machine learning project.
https://github.com/Unity-Technologies/ml-agents
But I'm just too lazy to use machine learning in my projects.
 
This is quite a nice video on NN being used for teaching the AI how to play snake:

It's a good example because it's a super simple AI, and you can visually see how the NN algorithm would work via the video: The Snake looks in 8 directions, and checks 3 variables:
  • Distance to Food,
  • distance to Tail,
  • Distance to wall
This (8x3 = 24) is all the input needed for the AI (As opposed to every on-screen pixel, like DeepMind was doing, which is not practical in GM). These inputs are fed into the algorithm, and the one of 4 potential outputs is the result.

This is really light processing: The heavy part would be with the generation aging (2000 simulations per generation..), but for an AI designed as simply as this that's still manageable by GML.
 
Top