Presenting Choice to the AI and Having It Reason

[David McMurdo]

Hello Everyone,

I'm still working on my strategy game with all the good advice that you've given me thus far, and this may be the last issue that I need to inquire about, but as ever, it's kind of tricky to explain.

I've attached a screenshot of my game here. See on the far left you have two groups of humans: a group of ten citizens, and a group of five spearmen. Every nation begins with these units. See that isolated block in the centre? That's a "foundation stone", and what each nation needs to do immediately is seek out such a foundation stone and then use their citizen unit to build a settlement around it. I've put a village in the room to the right just to show you what it looks like once a foundation stone has been settled—the stone becomes the centrepiece of whatever settlement is built there.

Now if I was enabling a human player to do this, I understand how I would do it: enable them to select the citizen unit, and have a choice in their options enabling them to settle. But in this case I'm trying to get the AI to do the same thing. However, I don't want to "fake it"—the AI should be making decisions just like a human player would. And my basic question is: what would that look like in code? This forum has already explained the use of states in AI decision making to me, but here I'm thinking more on the micro level. So I can have the AI in a state called "settle", say, but what would the code of that state look like?

Let's say that a group of citizens can not only settle, but also mine, or stand there dancing for no reason. How would I present such a choice to the AI in code, getting it to recognise that, at the present time, what they need to do is settle? I'm not asking anyone to write the code for me, I'd just appreciate some pointers as to what "features" of GML I should be working with, bearing in mind that there might be good reason to avoid claiming the first foundation stone they come upon (maybe it's right next to a monster lair or something).

In a related issue, there are going to be multiple competing nations in any given game, and at the beginning they are all seeking to settle. So obviously the AI would need a way of knowing whether or not a foundation stone had already been claimed. Am I right in thinking that the simplest way would be to have states for the foundation stone instances themselves: "claimed" and "unclaimed"? The AI would then have a way of distinguishing between those foundation stones that are available to them and those that aren't.

 

[Nidoking]

However, I don't want to "fake it"—the AI should be making decisions just like a human player would.

I'm not sure you understand what the "artificial" part of "Artificial Intelligence" means. It's all fake. All of it. AI is, and has always been, computer logic written to describe how the computer should make its decisions. Whatever you do to make a decision, that's what you have to tell the computer to do. If there's some situation it needs to consider, you have to program it to look for that situation and how to consider it. Most of the time, that's a heuristic function - a numeric score for the game state that will result from each choice. You're turning your game into math. That's the macro, the micro, and every level in between.

 

[David McMurdo]

I'm not sure you understand what the "artificial" part of "Artificial Intelligence" means. It's all fake. All of it. AI is, and has always been, computer logic written to describe how the computer should make its decisions. Whatever you do to make a decision, that's what you have to tell the computer to do. If there's some situation it needs to consider, you have to program it to look for that situation and how to consider it. Most of the time, that's a heuristic function - a numeric score for the game state that will result from each choice. You're turning your game into math. That's the macro, the micro, and every level in between.

What I meant was that I know how to get the citizen unit to move to the foundation stone and have a village appear, so that to the player it looks like the AI is playing the game. That's what I meant by "faking it".

 

[Stra]

Have you considered having something like a hierarchy of a nation's needs?

Highest need: settle - allocate all the citizens to go seek and claim a stone

If that need is met, go one level lower: constructing stuff, picking berries, plowing fields. That's 1/3 each and allocate the proper number (plus some random) of citizens to do that.

This would permit you to dynamically shift those needs and have different AI 'personalities'.

there might be good reason to avoid claiming the first foundation stone they come upon

That I think complicates things quite, you'd first need to define what those 'reasons' are and how can the AI perceive its surrounding to check for those circumstances.

 

[David McMurdo]

Have you considered having something like a hierarchy of a nation's needs?

Highest need: settle - allocate all the citizens to go seek and claim a stone

If that need is met, go one level lower: constructing stuff, picking berries, plowing fields. That's 1/3 each and allocate the proper number (plus some random) of citizens to do that.

This would permit you to dynamically shift those needs and have different AI 'personalities'.

That I think complicates things quite, you'd first need to define what those 'reasons' are and how can the AI perceive its surrounding to check for those circumstances.

You know how games like Age of Empires and Empire Earth have different ages that nations move through? This game doesn't have that from the player's perspective, but I thought that maybe from the AI's perspective it might. So for example, maybe this starting period would be "nomadic"—perhaps I can create an AI state called "nomadic" in which they prioritise settling, as you say. Back in the day I used to be one to have a look at the scripts of such games, and AI behaviour was dependent upon the "age" they were in. How to handle this in code is really the crux of what I'm getting at.

When it comes to getting the AI to avoid certain foundation stones, I figured that I might be able to code it so that they will settle on the nearest foundation stone, UNLESS there is a threatening object within a certain distance of that stone. I think that would be simple enough. But then there would also have to be exceptions. Like maybe that foundation stone is the only one available, so despite the proximity of the threat, they have to use it.

 

[Mr Magnus]

There is a certain joke in the field of computer science in that all AI will boil down to new and ever more complex ways of writing a hundred if-else statements.

We can not answer what the code will look like for any given state, it's such an impossibly vague question. I write card games for a living and despite a lot of the games being more-or-less the same within reason often the AI's decision trees will look wildly different as different games prioritize different aspects to enough of a degree to make the AI for the last game completely useless. Even within trick-taking games which all boil down to pretty much the same gameplay you can have very nuanced differences.

For the settle state you can approach it half a hundred ways. It may start off with you sending off your scouts to all viable directions. Once any of them find a stone check if it is unclaimed and if there is something undesirable about that location. If it is desirable enough or if enough time has passed and nothing better has shown up claim it and move on to the next state.

For multiple actions you'll need to find a way to rank the priorities of your empire. If you can pick berries, mine iron, and build houses you must run over your empire and go "How many berries do I need? How much Iron do I need? What buildings do I need and what do they cost?". You'll then have to split your villagers down to the tasks that have the most priority. If you have a berry farm automatically producing 50 berries per minute, and you only consume 25 berries per minute, then the "picking berries" action would have a value of 0: you do not need more berries. If you have no army and each army unit costs 4 iron, but you have 1 iron, and you need 50 units, then mining iron or building buildings that produce iron should have a high priority because it's an urgent task: you *need* iron right now.

This doesn't have to be much more complicated than a bunch of if statements checking every action that is relevant for this state and seeing if this action is something you need to do right now.

 

[David McMurdo]

There is a certain joke in the field of computer science in that all AI will boil down to new and ever more complex ways of writing a hundred if-else statements.

We can not answer what the code will look like for any given state, it's such an impossibly vague question. I write card games for a living and despite a lot of the games being more-or-less the same within reason often the AI's decision trees will look wildly different as different games prioritize different aspects to enough of a degree to make the AI for the last game completely useless. Even within trick-taking games which all boil down to pretty much the same gameplay you can have very nuanced differences.

For the settle state you can approach it half a hundred ways. It may start off with you sending off your scouts to all viable directions. Once any of them find a stone check if it is unclaimed and if there is something undesirable about that location. If it is desirable enough or if enough time has passed and nothing better has shown up claim it and move on to the next state.

For multiple actions you'll need to find a way to rank the priorities of your empire. If you can pick berries, mine iron, and build houses you must run over your empire and go "How many berries do I need? How much Iron do I need? What buildings do I need and what do they cost?". You'll then have to split your villagers down to the tasks that have the most priority. If you have a berry farm automatically producing 50 berries per minute, and you only consume 25 berries per minute, then the "picking berries" action would have a value of 0: you do not need more berries. If you have no army and each army unit costs 4 iron, but you have 1 iron, and you need 50 units, then mining iron or building buildings that produce iron should have a high priority because it's an urgent task: you *need* iron right now.

This doesn't have to be much more complicated than a bunch of if statements checking every action that is relevant for this state and seeing if this action is something you need to do right now.

So then it could be as something as simple as the AI recognising that it has no settlements and that it therefore needs to build one, if it has the citizens to do so. But what would such "resources" be present in the game as? If a nation has to concern itself with "settlements", "gold", and "food", say, would those be global variables?

I actually don't have a scouting unit at the current time because I'm not sure how to get the AI to "discover" things. I assume it would be a matter of somehow having the AI acknowledge the presence of something if it comes within a certain range of one of their units.

 

[Mr Magnus]

So then it could be as something as simple as the AI recognising that it has no settlements and that it therefore needs to build one

Yes, for instance. It knows it needs a settlement to do *anything* of value.

Do I have a settlement?
No? Well, do I know of a founding stone? No? Search for one. Yes? Is it a good founding stone? Yes? Settle it. No? find a better one, and so on and so forth.
.

Resources should be kept as close to the AI as possible. After all multiple AI's in one game can't all have the same bank. They need a way to separate their piles of gold from both each other and the player. Try to avoid global variables unless you absolutely need them: data should be as close to the thing that is using that data as possible and no further up.

 

[Director_X]

You can use a weighted average system to assign numerical priority to each need.

If certain NEED ie "Do I own atleast ONE village?" is >= 100, then it's a DIRE need,
as opposed to "Do I need to dance?" = low priority = 1;

 

[David McMurdo]

Yes, for instance. It knows it needs a settlement to do *anything* of value.

Do I have a settlement?
No? Well, do I know of a founding stone? No? Search for one. Yes? Is it a good founding stone? Yes? Settle it. No? find a better one, and so on and so forth.
.

Resources should be kept as close to the AI as possible. After all multiple AI's in one game can't all have the same bank. They need a way to separate their piles of gold from both each other and the player. Try to avoid global variables unless you absolutely need them: data should be as close to the thing that is using that data as possible and no further up.

You can use a weighted average system to assign numerical priority to each need.

If certain NEED ie "Do I own atleast ONE village?" is >= 100, then it's a DIRE need,
as opposed to "Do I need to dance?" = low priority = 1;

I know what you mean by "close to the AI", but I'm not sure how much can be kept close, or else don't know how to do it. So let's say in a certain nation's object I have some local variables representing how much food and gold they have. If that nation claims a settlement, wouldn't it be code within that settlement's object adding however much food and gold it provides to the nation that owns it? I'm still relatively new to Game Maker, but I thought that what distinguished a local variable from a global variable was that the former are only mutable within the object within which they are created, which is why I assumed I would need to work with global variables.

I also have a question that is more of a mathematical one (mathematics being something I've always been terrible at). A quarter of the population within any settlement will be available for distribution among a variety of tasks. So I would presumably have a local variable for the population that I tell the game to quarter for use by the AI. My first question is, what would that calculation look like in GML, and how would I get it to round down (or does GML handle such things in a specific way?) Obviously there will be occasions where a quarter of a population will not be a whole number.

 

[Mr Magnus]

I thought that what distinguished a local variable from a global variable was that the former are only mutable within the object within which they are created, which is why I assumed I would need to work with global variables.

Game maker does not do encapsulation. Any instance can access the variables of any other instance, no restrictions about it. There is a reason why the <instance_id>.variable syntax is a thing.
There are three types of variables in game maker.

1. Local variables (the ones declared with var) are temporary and belong to that script or event. They expire as soon as that script or event end, and thus are good for temporary calculations where you don't care if the variable is lost, it's just meant to be there for that single script.

2. Instance variables are variables that belong to certain instances, and lives and dies with that instance. As soon as the instance is destroyed the variable goes away. This allows multiple objects to all have variables that have the same name without it conflicting with each other. While usually the instance itself is the one that will commonly use it *any* other instance or script or function in the game can access it. You could for instance have a "o_ctrl" object that manages the game state, and other objects could go o_ctrl.game_state, or if there are many enemies and you need the health of one of them you can use any of multiple functions to get the instance id of that enemy and then access their health. Note that I assume this is what you really mean when you say "local variable", but in Game Maker local variables are a different thing, they are the variables that go away as soon as the script is done.

3. Global variables. These are eternal: once declared they are not going away. They belong to no instance or object, they are just a permanent part of your game memory and live in the void. They essentially belong to an always-existing object called global which you can not destroy, create, nor meaningfully alter. Global variables should really only be used if it makes sense to have some variable that isn't tied to any object and just exists as a fact of life in this game.

There is nothing stopping you from storing the resources of the AI nations in a "AI nation" instance and then having other instances refer to that. You could also have a "bank" object that has a struct or array or something to store everyone's resources. The main thing is just that you have an easy way to keep the resources of different nations separate.

A quarter of the population within any settlement will be available for distribution among a variety of tasks. So I would presumably have a local variable for the population that I tell the game to quarter for use by the AI. My first question is, what would that calculation look like in GML, and how would I get it to round down (or does GML handle such things in a specific way?) Obviously there will be occasions where a quarter of a population will not be a whole number.

GML has a whole section on math functions. floor, round. ceil, and other rounding functions exist for you to make sure you only ever get integers out of your calculations.

And again, that depends entirely how you set things up. you can handle it half a million ways: be it a list of all the pops assigned to different tasks, or a series of variables that tell you who is doing what, to just iterating over every pop in the game and ask "so, uh, what are you doing now?".

 

[David McMurdo]

Okay, thanks, that's really helpful. But what about writing rules? What I mean is this: to build a settlement you have to move a civilian unit to an unoccupied foundation stone and tell them to found a settlement. In what would that "rule" be contained? We've been talking about how to get the AI to focus on different tasks, but what are those tasks bound up in? For example, Game Maker has inbuilt functions. I figure that I have to create my tasks as something like that, so when I'm giving the AI the possibility to build a settlement in code, I can just write something like "build_settlement" and it will understand the rule. But what would "build_settlement" be? Some kind of custom function that is stored in a script somewhere? Given that such rules would be true for all AI nations. I hope what I'm getting at is clear.

 

[Nidoking]

Hello! Welcome to the concept of "game design"! This is the point in making a game where you understand the questions you need to ask, and now you have to figure out how to answer them. It's a good time to pivot a bit and start doing some tutorials on more intermediate concepts, or at least studying design patterns so you have a general idea of how game logic is put together. There's no single best answer for questions like these - if there were, I don't think games would exist at all. This is the point where the decisions you make will make your game a unique thing from all the other games out there. In short, good questions, but I think you're getting out of the scope of what any forum will be able to provide in a meaningful way.

That said, it sounds like what you're looking for is a state machine pattern. Your AI player will need to examine the situation and choose a state to be in. You come up with the list of states, the criteria for choosing a state, and what actions the AI will take in each state. Things like when it will check whether it should change states, how specific the states will be, etc. are all the pieces of design you need to come up with. More complex logic, more work, but the more responsive and interesting the result will be. Once you understand the concept of a state machine, you'll be in a better position to build your game around it.

 

[David McMurdo]

Hello! Welcome to the concept of "game design"! This is the point in making a game where you understand the questions you need to ask, and now you have to figure out how to answer them. It's a good time to pivot a bit and start doing some tutorials on more intermediate concepts, or at least studying design patterns so you have a general idea of how game logic is put together. There's no single best answer for questions like these - if there were, I don't think games would exist at all. This is the point where the decisions you make will make your game a unique thing from all the other games out there. In short, good questions, but I think you're getting out of the scope of what any forum will be able to provide in a meaningful way.

That said, it sounds like what you're looking for is a state machine pattern. Your AI player will need to examine the situation and choose a state to be in. You come up with the list of states, the criteria for choosing a state, and what actions the AI will take in each state. Things like when it will check whether it should change states, how specific the states will be, etc. are all the pieces of design you need to come up with. More complex logic, more work, but the more responsive and interesting the result will be. Once you understand the concept of a state machine, you'll be in a better position to build your game around it.

Thanks. Okay then, please let me be cheeky and ask a separate, but very specific question (I don't want to start another new thread).

I've put a lot of work into the art assets for this game, and units will be able to walk in eight directions. There is a walking animation for all eight directions, and an idle animation for all eight directions for every unit. Here is the code for one of the units.

if (direction >= 337.0 or direction <= 22.0 and speed >= 0.1)
    {
        sprite_index = sprAtlanteanWarriorsE;
    }

else

if (direction >= 337.0 or direction <= 22.0 and speed = 0.0)
    {
        sprite_index = sprAtlanteanWarriorsEIdle;
    }

if (direction >= 22.1 and direction <= 67.1 and speed >= 0.1)
    {
        sprite_index = sprAtlanteanWarriorsNE;
    }

else

if (direction >= 22.1 and direction <= 67.1 and speed = 0.0)
    {
        sprite_index = sprAtlanteanWarriorsNEIdle;
    }

if (direction >= 67.2 and direction <= 112.2 and speed >= 0.1)
    {
        sprite_index = sprAtlanteanWarriorsN;
    }

else

if (direction >= 67.2 and direction <= 112.2 and speed = 0.0)
    {
        sprite_index = sprAtlanteanWarriorsNIdle;
    }

if (direction >= 112.3 and direction <= 157.3 and speed >= 0.1)
    {
        sprite_index = sprAtlanteanWarriorsNW;
    }

else

if (direction >= 112.3 and direction <= 157.3 and speed = 0.0)
    {
        sprite_index = sprAtlanteanWarriorsNWIdle;
    }

if (direction >= 157.4 and direction <= 202.4 and speed >= 0.1)
    {
        sprite_index = sprAtlanteanWarriorsW;
    }

else

if (direction >= 157.4 and direction <= 202.4 and speed = 0.0)
    {
        sprite_index = sprAtlanteanWarriorsWIdle;
    }

if (direction >= 203.5 and direction <= 248.5 and speed >= 0.1)
    {
        sprite_index = sprAtlanteanWarriorsSW;
    }

else

if (direction >= 203.5 and direction <= 248.5 and speed = 0.0)
    {
        sprite_index = sprAtlanteanWarriorsSWIdle;
    }

if (direction >= 249.6 and direction <= 294.6 and speed >= 0.1)
    {
        sprite_index = sprAtlanteanWarriorsS;
    }

else

if (direction >= 249.6 and direction <= 294.6 and speed = 0.0)
    {
        sprite_index = sprAtlanteanWarriorsSIdle;
    }

if (direction >= 295.7 and direction <= 336.9 and speed >= 0.1)
    {
        sprite_index = sprAtlanteanWarriorsSE;
    }

else

if (direction >= 295.7 and direction <= 336.9 and speed = 0.0)
    {
        sprite_index = sprAtlanteanWarriorsSEIdle;
    }

Now all of that works perfectly—when the unit is moving in any direction, it plays the correct movement animation, and when the unit stops, it plays the idle animation for the direction it is facing when it is stopped. That is, except for when the unit is walking east "below" the 360 degree mark. For some reason, when the unit stops having walked in that direction, the east idle animation doesn't trigger, and the units continue marching on the spot. I don't understand this because the same code works for all other directions, and I hope that perhaps a fresh eye could catch something that I'm not catching in the code for the east direction that would cause the idle animation to fail to trigger. I tend to name everything longhand, so I think the sprite names speak for themselves, but I suspect that it might be something I'm missing in the way that Game Maker handles direction. I doubt it's a coincidence that the one direction that doesn't work happens to be the one direction that goes from a "high degree" to a "low degree" and that therefore uses an "or" statement instead of an "and" statement.

Jeez. Just trying to explain these issues is a task.

EDIT: And if you have any tips for doing what I'm doing in a better way (perhaps the way I've portioned the directions is off), please feel free to share. I'm eager to learn.

 

[Nidoking]

Try putting parentheses around the or part, since that's the part you want to evaluate first. The and may be binding more tightly than the or, resulting in if (direction >= 337.0 or (direction <= 22.0 and speed = 0.0))

I would also use the debugger to look at the speed value, since it might be 0.0000001 or something like that.

 

[Slow Fingers]

I would use a weighted system of importance for stuff, if you want your AI to look 'intelligent'.
What is most important to move forward in your game at the macro-level? Build a new town? Start a war? Mine resources?
What will determine the order of priority of these things? When do these priorities change?
Take an example: say an AI in Starcraft has got it's ass kicked and lost all of it's SCVs. It has 75 minerals left. It would be pretty stupid for it to build a marine, and not try to get a new SCV out, would it?

All in all, it seems you just don't have your logic flow nailed down yet, which makes it a very bad idea to try to start coding it right away, if you ask me.

 

[David McMurdo]

Try putting parentheses around the or part, since that's the part you want to evaluate first. The and may be binding more tightly than the or, resulting in if (direction >= 337.0 or (direction <= 22.0 and speed = 0.0))

I would also use the debugger to look at the speed value, since it might be 0.0000001 or something like that.

Cheers—adding more parentheses worked, I just had to render it slightly differently:

if ((direction >= 337.0 or direction <= 22.0) and speed >= 0.1)
    {
        sprite_index = sprAtlanteanCitizensE;
    }

else

if ((direction >= 337.0 or direction <= 22.0) and speed = 0.0)
    {
        sprite_index = sprAtlanteanCitizensEIdle;
    }

Alright, now time to dig in to this AI business and get it going. Thanks for everyone's contribution here; I've bookmarked this thread for reference.

 

[trippity]

Hello friend,

I'm a Machine Learning engineer dabbling in some game making so I may be able to offer some guidance. The route you take will ultimately depend on what kind of 'experience' you want your AI to give the player.

Classical game 'AI':
This is largely rule based, specifically, state machines integrated with decision trees with an over arching manager controlling states based on in game 'global' variables. By global, I mean what is happening currently in game across all players.
Pros:

• Relatively easy to implement. Start with a white board and define a clear scope of how many states you want, how many decisions per state and what are the triggers for getting in and out of states.

Cons:

• Very rigid, rules will need to be balanced as to not give an unfair advantage/repeating experience
• Can become cumbersome and difficult to manage depending on complexity of the rules & weights you place on actions.

Recommendations - Start simple, experiment, then come back to your project requirements.

• Start with 2 states, fight & collect and have a manager which can detect if home base is under attack. If 'home base' taking damage then switch state to fight else collect.
• Build on the above states and expand your managers reach with more complex requirements for state changes.
• As others have mentioned you will need to define a weighting system to make the 'experience' more dynamic. A quick example for attack state could be: Attack_Priority = Delta_Homebase_Health/Delta_Time. In English, your attack priority is inversely proportional to how quickly you're losing health. You could then check which state has the largest number and switch to said state...
• Repeat for decisions in each state ie. each state will then have its own sub manager that decides on actions in that state (if you want to make it that complex).

Food for thought:

• One thing to note that I haven't seen mentioned above (apologies if you have) is the AI rules suggested largely focus on its own variables: "Do we have enough resources, are we under attack" etc. This can be loosely described as a "first order" AI. Its reactive only and will most likely be beaten by the player every time. The AI needs to also take into account the player movements, "how much territory do they cover, how much resources do they have, what units have we encountered". This is what will make the AI feel dynamic and alive.
• To achieve this, you could simply load your 'stats' into a variable the AI can see every 30 seconds for example..which then feeds into your managers weight system.
• In the end its just a matter of each unit and your units feeding information UP from their states and into your over arching manager. Google voting/weighting systems for ideas, it doesn't have to be complicated.

'Modern' AI:
Now we step into the world of 'true' AI (take that with a pinch of salt), namely Neural Networks (they work well in games).
Pros:

• If built and trained correctly, they can offer great game-play experiences (see Elon Musks Dota 2 AI) also look at 'Reinforcement Learning'
• The 'agents' (your AI fellows) will define their own 'decision trees' and switch their own 'states' = Saves you having to code them explicitly. See AI Plays hide and seek

Cons:

• Can be very difficult to implement and train. You will need to train your agents across many different settings (maps & enemies) to develop a 'general' intelligence.
• The player ultimately doesn't care how fancy your AI is and you run the risk of your AI doing stupid things and ruining the experience.
• Absolutely no way of debugging behavior..

Conclusion:
If you want to cheat, I would suggest walking outside and finding an ants nest. The best code is written by nature. In the end all that engineers/AI researchers etc are doing is replicating what they see in the wild..
Study their behavior and you will quickly see that they achieve the behavior you want in your game - Replicate their triggers and messaging. Proximity sets off attacks, feeding pheromones = X,Y coordinates for resources etc.

Hope this helps


EDIT:
In terms of what 'features' you will need, I would say little to none. Perhaps the most complex would be ds_grids to store information in. However could easily just put data in arrays - All you're doing is handling numbers.
Example use case:


Then you could have your manager keep tabs on the strength index for example - The index being a weighted average of columns B,C,D &E.

Again we can use the derivative strength increase/time to signal a rising force in the game. So if Tribe 1 is currently leading and they notice Tribe 3 starts gaining fast = Drop what you're doing and attack Tribe 3.

 

[RefresherTowel]

Hello friend,

I'm a Machine Learning engineer dabbling in some game making so I may be able to offer some guidance. The route you take will ultimately depend on what kind of 'experience' you want your AI to give the player.

Classical game 'AI':
This is largely rule based, specifically, state machines integrated with decision trees with an over arching manager controlling states based on in game 'global' variables. By global, I mean what is happening currently in game across all players.
Pros:

• Relatively easy to implement. Start with a white board and define a clear scope of how many states you want, how many decisions per state and what are the triggers for getting in and out of states.

Cons:

• Very rigid, rules will need to be balanced as to not give an unfair advantage/repeating experience
• Can become cumbersome and difficult to manage depending on complexity of the rules & weights you place on actions.

Recommendations - Start simple, experiment, then come back to your project requirements.

• Start with 2 states, fight & collect and have a manager which can detect if home base is under attack. If 'home base' taking damage then switch state to fight else collect.
• Build on the above states and expand your managers reach with more complex requirements for state changes.
• As others have mentioned you will need to define a weighting system to make the 'experience' more dynamic. A quick example for attack state could be: Attack_Priority = Delta_Homebase_Health/Delta_Time. In English, your attack priority is inversely proportional to how quickly you're losing health. You could then check which state has the largest number and switch to said state...
• Repeat for decisions in each state ie. each state will then have its own sub manager that decides on actions in that state (if you want to make it that complex).

Food for thought:

• One thing to note that I haven't seen mentioned above (apologies if you have) is the AI rules suggested largely focus on its own variables: "Do we have enough resources, are we under attack" etc. This can be loosely described as a "first order" AI. Its reactive only and will most likely be beaten by the player every time. The AI needs to also take into account the player movements, "how much territory do they cover, how much resources do they have, what units have we encountered". This is what will make the AI feel dynamic and alive.
• To achieve this, you could simply load your 'stats' into a variable the AI can see every 30 seconds for example..which then feeds into your managers weight system.
• In the end its just a matter of each unit and your units feeding information UP from their states and into your over arching manager. Google voting/weighting systems for ideas, it doesn't have to be complicated.

'Modern' AI:
Now we step into the world of 'true' AI (take that with a pinch of salt), namely Neural Networks (they work well in games).
Pros:

• If built and trained correctly, they can offer great game-play experiences (see Elon Musks Dota 2 AI) also look at 'Reinforcement Learning'
• The 'agents' (your AI fellows) will define their own 'decision trees' and switch their own 'states' = Saves you having to code them explicitly. See AI Plays hide and seek

Cons:

• Can be very difficult to implement and train. You will need to train your agents across many different settings (maps & enemies) to develop a 'general' intelligence.
• The player ultimately doesn't care how fancy your AI is and you run the risk of your AI doing stupid things and ruining the experience.
• Absolutely no way of debugging behavior..

Conclusion:
If you want to cheat, I would suggest walking outside and finding an ants nest. The best code is written by nature. In the end all that engineers/AI researchers etc are doing is replicating what they see in the wild..
Study their behavior and you will quickly see that they achieve the behavior you want in your game - Replicate their triggers and messaging. Proximity sets off attacks, feeding pheromones = X,Y coordinates for resources etc.

Hope this helps


EDIT:
In terms of what 'features' you will need, I would say little to none. Perhaps the most complex would be ds_grids to store information in. However could easily just put data in arrays - All you're doing is handling numbers.
Example use case:
View attachment 43542

Then you could have your manager keep tabs on the strength index for example - The index being a weighted average of columns B,C,D &E.

Again we can use the derivative strength increase/time to signal a rising force in the game. So if Tribe 1 is currently leading and they notice Tribe 3 starts gaining fast = Drop what you're doing and attack Tribe 3.

I love this breakdown, it's great to hear from someone actually working in machine learning! I do have to point out though that machine learning for AI in a game is very often considered overkill. The interesting thing with AI/player interactions is that players often read waaay more into AI actions than actually exist and so less is more in a certain sense, we can program (relatively) simple AIs and they will usually fool the player. The player will build stories out of the actions they see without having an understanding of why that action is taking place and very often, the reason why the AI is doing a particular thing at a particular time has very little to do with some advanced routine based around combating the exact way the player is playing. It's far more likely to be a simple heuristic that comes from your classical game AI example. I think it's usually better (most especially for people who are just delving into the world of AI) to start working from the simpler premise of "I will program in a few rules for the AI to follow blindly" than building a machine learning system that is meant to learn strategies and eventually play against a player. The player is not terribly likely to significantly appreciate the difference between the two, and one is muuuch less complicated than the other, and considering our most precious resource as developers is time, we should take the simpler action that yields roughly the same results whenever possible.

That being said, inserting machine learning into games as an AI manager is definitely an interesting and fruitful area to explore as designers and programmers and I don't want my response to be a "rain on your parade" kind of thing. I just don't think it's super helpful when someone is asking basic questions about AI in general.

 

[David McMurdo]

Hello friend,

I'm a Machine Learning engineer dabbling in some game making so I may be able to offer some guidance. The route you take will ultimately depend on what kind of 'experience' you want your AI to give the player.

*SNIP*

Thank you very much for that post. I've just sat down to get grinding on the game for the rest of the day, so your timing was perfect. I'll be using your post as a guide while I work.

At this point I think it might be useful if I say something about the premise of the game and why player interaction isn't relevant for now.

I'm creating a god/strategy game hybrid. The game will look like an Age of Empires type strategy game for all intents and purposes, but actually, the player does not control any nation, but is rather an archangel overlooking all of them. All nations featured are mythical nations such as Atlantis, Lemuria, Mu, etc, and in each session there will be a countdown to a cataclysm in which all nations will be destroyed. As an archangel, the player's mission is not to ensure the supremacy of any one nation, but to save as many souls in the world as possible before the inevitable destruction. However, the only entities in the world that the player will have direct control over are those souls that have awakened to the reality of the divine, whichever nation they emerge from. The player will use such enlightened souls to influence the behaviour of nations in various ways. For example, perhaps one of the nations is behaving hyper aggressively, and so the player has one of their servants encourage that nation to adopt the worship of a peaceful deity, which will have a cooling effect on their diplomatic policy. Sometimes the interactions will be very "Old Testament": there will be a mechanic whereby settlements can grow decadent, (and this is more likely to happen with civilizational development, thus hopefully doing something to avoid the old "late game ease" that affects most strategy/god games). It may be necessary for the player to use their god powers to destroy such a city as a warning to others not to follow in their path. The player themselves will answer to a Divine Council that will bestow short-term tasks and things like that. God games thus far have given the player what is ultimately a very human agenda—to ensure the material supremacy of a particular people, and I'm really in love with the idea of giving the player an unearthly agenda that may coincide or conflict with the worldly agendas of the AI nations. I think this idea has a lot of potential. Besides my own interest in spirituality and prehistory, I was always really inspired by games like Actraiser, the Seven Kingdoms games, and the Dominions series. But I want to go further than they did with their ideas.

Of course, ideas are cheap, and this is no small undertaking, especially for a newbie like myself. I figured that the best way to approach this project was to start building this game as if I was making a conventional, late-nineties RTS for people to play and THEN develop the "god game layer" on top of it, because I want the strategy layer to be as authentic as possible. If I can have the AI playing out a strategy game that is played by coherent rules, then having the player influence those rules as a divine being will be a relatively simple affair. This is why I only have to be concerned with getting the AI to play the game; the player interacts with the world on a different "level" than the AI in view here. I've already planned how I intend the strategy game to function in terms of mechanics, and the art assets I need are already created, it's now just a matter of teaching the AI, your assistance with which I very much appreciate.

 

[trippity]

I love this breakdown, it's great to hear from someone actually working in machine learning! I do have to point out though that machine learning for AI in a game is very often considered overkill.

I absolutely agree, aside from the Dota example I haven't really seen a good implementation of it. Thus no recommendation to go down this avenue. I added the 'modern' component for the odd chance someone stumbles onto this thread looking for a place to start in that regard..might save someone having to create a post

Just to clarify, I never suggested that the manager be a machine learning algorithm (as mentioned, its overkill). A voting system based on index scores of sorts to just switch states would suffice.

EDIT:

@David McMurdo hopefully I haven't confused you.

To boil it all down. Pump all the game variables you need into one place (perhaps the grid?) calculate a weighted score for each tribe
In your 'manager' write some rules around:
-Score values
-How quickly the values change (like my threat example above)

Then you can create a feedback loop so weights change based on calculated scores and round and round it goes.

 

[RefresherTowel]

I added the 'modern' component for the odd chance someone stumbles onto this thread looking for a place to start in that regard..might save someone having to create a post

Definitely, I didn't see your post as useless or counter-productive. It's an area of game design/programming that isn't often broached and it's useful to have it broken down in the ways you did =) I think there's a lot of room for amazing advancements in game AI once machine learning matures a little more in terms of how it can relate to player experiences. I just wanted to make sure we're not sending beginners down the rabbit hole of machine learning without them realising that most of the "smart" AI they encounter in games doesn't stem from that awesome, but deep and slippery, hole.

 

[David McMurdo]

I think I understand what I should be aiming for, but what I'm a little confused about at the moment is more on the micro level.

So let's say I have only two objects: an AI controller object for the Atlanteans, and an object for an Atlantean settler unit.

Let's say that the AI controller was in an "expand" state due to the present state of affairs. Would the command for the specific actions that the settler should take be contained within the settler object or the AI object? Can one object directly tell another object how to behave?

Could I design it so that each AI nation has different states, and each unit type also has states? Can states evoke states? Let's say that a civilian unit can both be in a "settling" state or a "mining" state; if the AI controller is set to an "expand" state and therefore seeks to settle, could that state dictate the state of the civilian unit?

I'm essentially seeking clarification on how the AI gives orders to existent instances; whether the code to control those instances should all be contained within the AI object for the relevant nation, or whether every units object should have its own states.

 

[RefresherTowel]

I think the answer is it can go either way. You could decide to have the AI manager controlling each individual unit with their specific tasks and have the units simply be "images" for the ideas of the controlling agent, or you could choose to have the units have their own states, with the AI manager deciding which state the units should move into. My personal preference for that sort of thing would be to have the units have their own states. My reasoning for this is the AI manager can act as a kind of "getter" and "setter", or an interface rather. You can have different units react differently to the same "state" that the AI manager decides is correct, and the state of the units is separate from the state of the manager. This gives you a bit of a buffer between decision and outcome that might be useful in particular circumstances. Generally, I like to break things apart into their components and this is one way of achieving that.

That being said, there's not really a game changing reason I can think of as to why I would pick one over the other. Maybe there might be some performance benefits to one or the other, but I would have to do some profiling for that. And I would only do that profiling if the modus operandi I chose was actually running slowly enough to be noticeable.

EDIT: Also, I would like to reiterate Nidoking's point previously. This kind of thing isn't decided. Programmers generally don't have a specific way of achieving these kinds of things. We each approach it with our background knowledge and a vague sense of what we are aiming to achieve and we iterate and update as we run into problems. There's no one correct solution or answer. Try to do it, fail, try to do it a different way, maybe fail, try to do it another different way, perhaps that works, etc. That's how I would approach the problem. Of course, reading the literature and learning from the past is totally correct, but no-one here is going to be able to offer the absolute solution to the problems you are facing (beyond the general solutions that have been offered already).

 

[David McMurdo]

I think the answer is it can go either way. You could decide to have the AI manager controlling each individual unit with their specific tasks and have the units simply be "images" for the ideas of the controlling agent, or you could choose to have the units have their own states, with the AI manager deciding which state the units should move into. My personal preference for that sort of thing would be to have the units have their own states. My reasoning for this is the AI manager can act as a kind of "getter" and "setter", or an interface rather. You can have different units react differently to the same "state" that the AI manager decides is correct, and the state of the units is separate from the state of the manager. This gives you a bit of a buffer between decision and outcome that might be useful in particular circumstances. Generally, I like to break things apart into their components and this is one way of achieving that.

That being said, there's not really a game changing reason I can think of as to why I would pick one over the other. Maybe there might be some performance benefits to one or the other, but I would have to do some profiling for that. And I would only do that profiling if the modus operandi I chose was actually running slowly enough to be noticeable.

EDIT: Also, I would like to reiterate Nidoking's point previously. This kind of thing isn't decided. Programmers generally don't have a specific way of achieving these kinds of things. We each approach it with our background knowledge and a vague sense of what we are aiming to achieve and we iterate and update as we run into problems. There's no one correct solution or answer. Try to do it, fail, try to do it a different way, maybe fail, try to do it another different way, perhaps that works, etc. That's how I would approach the problem. Of course, reading the literature and learning from the past is totally correct, but no-one here is going to be able to offer the absolute solution to the problems you are facing (beyond the general solutions that have been offered already).

I appreciate that, though it does rather surprise me; I'd have thought that after three decades of real-time strategy games there would very much be a "template" for how to do things, but maybe such templates are all "in house" (such as Ensemble Studios building on the code of Age of Empires 1 for Age of Empires 2). I just wanted to make sure that my idea of AI states evoking object states wasn't a dead end for some reason that I can't foresee due to my inexperience in Game Maker before I began working on it. I thought it would be a good idea for exactly the reason you mentioned: it just seems neater and more intuitive to have the unit objects contain the possible behaviours of those units, rather than having all of those behaviours contained within a single, separate AI controller.

 

[Nidoking]

Like I said before, this is the "artificial" part of artificial intelligence. The precis is that AI, essentially, is the process of turning the state of a game into a mathematical problem that a computer can solve, either by determining an answer to the problem or, more commonly, by optimizing the result of the calculation to get the best outcome. The process of determining the maximum value of a function based on constraints is a well-understood mathematical model, so the real work in AI is in determining how to translate the game state into that model. This is called a "Heuristic function", and it will be the backbone of absolutely any AI you write. Even the neural net needs to have a number to optimize, even if you're not explicitly writing the logic that uses it. The further specifics of how the AI uses those numbers to make decisions, including things like how many steps ahead to plan and how to choose between moves that look equally good, are just further tailoring you can add once you have the heuristic function squared away.

A lot of what you're talking about in terms of states could be incorporated into the heuristic. For example, if choosing a place to settle has to come before anything else, then not having a settlement would be a big negative in the heuristic value, and a game state with a settlement would have a much higher rating than a state without a settlement. But you could also have the decision algorithm use broad stages without looking at the numbers directly. The first step could be "make a settlement", and you use a function that determines the value of each location compared to the time it takes to settle there and the potential value of searching for better locations. Then you can switch to a heuristic tailored to the next stage. All of it boils down to figuring out what considerations you, as a human, think about when deciding what to do, and then finding explicit ways to describe that in logic, where all information is perfect but intuition can't exist. Everything you'll learn about AI is essentially going to be just specifics of how to do that.

 

[Mr Magnus]

A common strategy in the "how does the AI do stuff" camp is also just "The same way the human does stuff, but via their own interface".

Like you're wondering how the AI controller tells the settler to go settle: can objects control other objects? The answer there could be (but does not have to be) "well, the human clicks the AI settler and clicks where they want the settler to go. That triggers some code in the settler to walk to that spot". If you are clever on how you set up your systems there is nothing that prevents you from going "The AI controls the settler by 'clicking' the settler and then 'clicking' where they want the settler to go, and the settler will trigger the exact same walking code as the human triggered with its clicks". Like, you could have a walkTo(units, _x, _y) function in some script. When the human clicks somewhere on the map something in the game goes walkTo( selectedUnits, mouse_x, mouse_y) . If the AI wants to do it it *too* could call walkTo(unitsOfInterest, destination_x, destination_y); . Repeat for everything you want the AI to be able to do: just make a version of the human interface that the AI can work with for its own team.

In my job the AI controller and the Human controller are both derived from the "Player" class, and they both use the exact same interface. The only difference is that the AI decides on its own what functions to call, while the human gets fancy buttons that trigger those calls. Under the hood the game can't tell the difference between the AI and the human, they interact with it the same way. All the game knows is that the current player called "drawCard()" and that player will get their card no matter if it has a brain of flesh or metal.

However there are no dead ends. Even if you'd pick the worst possible strategy the only dead end is your own skill and patience. You can make anything work, and the main stopping block is the decisions you make. GameMaker at no point will go "Sorry buddy, that's not how we do things in the RTS world". A lot of these low-level decisions on implementation are always going to be dependent on how you've designed the rest of your code.

 

[RefresherTowel]

I'd have thought that after three decades of real-time strategy games there would very much be a "template" for how to do things

You'd have thought that certainly, but there's a bazillion implementations of AI pathfinding, which has been a problem for decades in games, and none of them are "the best" way to go about things. It all comes down to what you specifically want to happen. Each algorithm has its benefits and drawbacks. General AI for a genre of game are the same as this, but a lot more complex. Whenever you want something to act in a "human" way, you are going to run into the problem that humans aren't computers and computers aren't human. Programmers have developed a number of different approaches to this problem but, as I said just before, they all are good when looked at from particular POV's and bad when looked at from others. Generally, we try to boil things down to numbers, as that is what a computer "thinks" in. How you make these numbers and what gets incorporated and what gets neglected when it comes to the "number" are decisions that have to be made from the design point of view. I think most problems in this area come from trying to think too "humanly" and most of the replies here are trying to get you to think "computerly". You need to boil down the game into strategic decisions which you would make as a player, assign a value to those strategic decisions and then give that number to an AI that acts on a set of procedures that you have written for each number. This is much easier said than done in reality, but it is the way things are done. There's no magic sauce to getting it done, and I imagine if you saw the source code from Ensemble Studios it would pretty much be what has been said in this thread but more advanced because it has had a bunch of experienced technical engineers working on it, rather than a single person programming in GML. There's not going to be a magic bullet that someone spouts that suddenly gives you an AI that responds to player decisions/world state without you specifically telling it what to do based around player decisions/world state.

 

[FrostyCat]

You started this topic with a false premise, and the more the topic progressed, the less coherent it became.

I want you to recognize the following:

• The more the field developed, the more the works grew away from each other, and the less room there was for any code-level commonality to exist between them.
• Computers cannot reason or know what you mean. They can only do what you tell it to do, and nothing more.

This rules out any sort of hope for an "automatic" or "universal" solution. So stop looking for them. You will not find one ready-made that will be automatically applicable to your work, and we can't and won't give one to you.

Not even something as obvious and ubiquitous as "dying when health goes to 0" is ever automatic. In every case there is a developer-specified check in a deliberately chosen location where it is applicable, with a developer-specified action to follow-up if it holds. That varies from project to project. Even the action of "dying" is diverse, ranging from a simple disappear, to simple animations, to physical simulations that spawned an entire field of study on its own (i.e. ragdoll physics). If something as basic as "dying when health goes to 0" can be this diverse, what makes you think any of the items you're asking for would have a definite solution?

In any case, I think you should put this "Age of Empires" idea down and return to the basics. Instead of learning decision-making in code via a strategy game AI that has many moving parts, begin with and focus on simpler ones like shooting at the player (direct computation), "dealer hits on 16 and stays on 17" (decision trees), or steering behaviours (acting on weighted sums). Then proceed when you don't feel the need to be bailed out anymore.

 

[David McMurdo]

You started this topic with a false premise, and the more the topic progressed, the less coherent it became.

I want you to recognize the following:

• The more the field developed, the more the works grew away from each other, and the less room there was for any code-level commonality to exist between them.
• Computers cannot reason or know what you mean. They can only do what you tell it to do, and nothing more.

This rules out any sort of hope for an "automatic" or "universal" solution. So stop looking for them. You will not find one ready-made that will be automatically applicable to your work, and we can't and won't give one to you.

Not even something as obvious and ubiquitous as "dying when health goes to 0" is ever automatic. In every case there is a developer-specified check in a deliberately chosen location where it is applicable, with a developer-specified action to follow-up if it holds. That varies from project to project. Even the action of "dying" is diverse, ranging from a simple disappear, to simple animations, to physical simulations that spawned an entire field of study on its own (i.e. ragdoll physics). If something as basic as "dying when health goes to 0" can be this diverse, what makes you think any of the items you're asking for would have a definite solution?

In any case, I think you should put this "Age of Empires" idea down and return to the basics. Instead of learning decision-making in code via a strategy game AI that has many moving parts, begin with and focus on simpler ones like shooting at the player (direct computation), "dealer hits on 16 and stays on 17" (decision trees), or steering behaviours (acting on weighted sums). Then proceed when you don't feel the need to be bailed out anymore.

I don't know what "false premise" you're referring to.

I understand perfectly well that there will be many things to achieve and many ways to achieve them, but I don't believe that it's an unreasonable expectation that maybe someone on this forum might have had some experience in trying to make a game of this nature and would therefore have useful input to give—a "template" as I put it before. For example, has Game Maker 2 ever been used to make an RTS before? If so, how did that person do it? Doesn't mean I think it's the only way, but it's an established way that would definitely be useful to know about. I'm not one to just leap to the forums immediately either—I read the manual and I look up tutorials. The problem I've found is that 95% of those tutorials seem to be about making a platform game or an RPG, and as a newcomer I find it hard to translate what those tutorials could teach me to what I want to achieve. So my only recourse has been to ask directly.

I did initially start work on multiple projects that are far less ambitious than this, but what I quickly found was that no matter how simple the project, it would still require a massive time investment in terms of art creation and coding, and if I'm going to make such an investment, I'd rather make it for the sake of a game that I'm truly interested in making. But maybe you're right and I should just create whatever sprites and experiment with the fundamentals.

 

[Slow Fingers]

I understand perfectly well that there will be many things to achieve and many ways to achieve them, but I don't believe that it's an unreasonable expectation that maybe someone on this forum might have had some experience in trying to make a game of this nature and would therefore have useful input to give—a "template" as I put it before.

What he means is that it's borderline useless, because it will be so dependent on your own code and architecture that no one will have done it the same. You got to learn the principles, and apply it to your particular situation.
There is no 'template'.
At this point it's like asking 'How to please my girlfriend?'
I mean....I could give you the general idea (that I'm already sure you know of), I could tell you what makes MY girl happy, but I sure as hell can't help you with yours unless I know her very well.
And as of now, no one's ever even seen a PICTURE of her, so it's kind of futile to ask for specific help.

Other than that, nothing beats reading a good ol' book on the subject
AI books

 

[Nidoking]

I did initially start work on multiple projects that are far less ambitious than this, but what I quickly found was that no matter how simple the project, it would still require a massive time investment in terms of art creation and coding, and if I'm going to make such an investment, I'd rather make it for the sake of a game that I'm truly interested in making.

I think this might be the core of your issue right here. The total time investment is not going to decrease if you choose to try to make your dream game immediately, and if you don't enjoy making smaller games, you're not going to enjoy making one big one. You're not going to build the complex game you want to make if you can't understand the simple fundamentals, and that's more than just object structure and code syntax. You need to be able to make pieces of things that will work together while being simple enough to manage. Game development is a broad spectrum of complexities, and you have to build up to whatever you want to do. The time you spend on simpler games isn't wasted - why are you making art that won't ultimately fit into your dream game? Why not harvest bits from what you've done previously to get a leg up in your next project? And the more small, simple things you understand how to do, the bigger a game you'll be able to make by putting them together. And if the time it takes to make a small, simple game is more than you're willing to put in, simply because the result isn't absolutely everything you've ever dreamed a game could be, what makes you think you'd be able to handle the exponentially greater investment it's going to take to make a game when you don't even have much understanding of what artificial intelligence IS? I took a whole college course in artificial intelligence - that's not the kind of scope of thing that can (or should) be covered in a forum thread alongside people asking why they can't set a variable in a sound resource. Heck, you could even build pieces of THIS game, just enough to make something that moves and is playable, and then start to build on top of that. Especially if what's already been posted here isn't enough for you, you're not going to get anything you would find useful no matter how many times you ask. Work on raising your own skills to the level where you understand what's been said here, and why it's been said, and what it's teaching you.