GameMaker Client/Server Architecture

B

BlaXun

Guest
Hello everybody,

I recently got a idea for a game I'd like to prototype... a sidescroller. However, as my previous game I want this one to be online at one point as well.

For my earlier games I had two seperate programs. The client (game) and the server.
What happened due to this is that the client had to inform the server bout position and such...and thus it wasn't reliable at all.
This time I want to learn from my mistakes, use UDP instead of TCP, have client-side prediction of positioning etc.

However, what I am struggling with is how to actually let the server know the correct position of each player... and it seems like the common approach for this is to actually have the game and server share the same codebase... even being the same compiled executable (even though hosting this on a server would require a server with a graphics card n such). With this setup the server would have to know the map for the player and it should then be able to know where players are positioned... I did not have this in my earlier games. It did work quite well I'd say, but I know its not the ideal solution :D

So, yeah. How does that idea of having client and server in the same executable sound? I mean, the server would not have to render anything ...maybe for debugging... but actually I'd even be happy with just a command-line interface (which GMS doesn't support afaik).

To me it sounds like a good start and it could solve some problems. "Unfortunately" the game will have multiple rooms and thus my idea would be to load all rooms on the same screen and have em overlapp, but the blocks that players in each room can interact with have something like a room-id so they know what room they belong to and on collisions I'd always have to check if the colliding block is actually within the same room as the player. It sounds like all these overlapping rooms could account to a lot of collision instances. A alternative approach would be to load rooms next to each other with a x and y offset to have a x,y origin at 0,0.... however this could result in having a very big gamefield on the server-side.

Thx in advance :)
It feels nice being back and thinking bout game-dev!

Edit:
Some links I found helpfull

https://forum.yoyogames.com/index.php?threads/some-multiplayer-design-questions.31689/

https://forum.yoyogames.com/index.php?threads/lag-friendly-real-time-networking-tips.47951/

https://medium.com/@meseta/2e5a98e4105f

https://net8floz.itch.io/fix-your-timestep-extension
 

The-any-Key

Member
the client had to inform the server bout position and such
This is how a multiplayer game work. A connection is just a line between games that allow you as the coder to send information. And youyas the coder must handle the data. As another coder said: multiplayer games are like a correspondence chess game. Games are never "connected" and sync automatically.

To have the server and client in the same app just make sense. Else you need to swap between projects.

UDP is the way to go. But you need to make sure you only send around 700 bytes per packet. Else some routers block them. You also need to handle the packet loss manually. But you can add a id id for each packet and if you expect packet 5 but got 7. You can request packet 6 again.
 
B

BlaXun

Guest
This is how a multiplayer game work. A connection is just a line between games that allow you as the coder to send information. And youyas the coder must handle the data. As another coder said: multiplayer games are like a correspondence chess game. Games are never "connected" and sync automatically.

To have the server and client in the same app just make sense. Else you need to swap between projects.

UDP is the way to go. But you need to make sure you only send around 700 bytes per packet. Else some routers block them. You also need to handle the packet loss manually. But you can add a id id for each packet and if you expect packet 5 but got 7. You can request packet 6 again.
Thx for the reply. I had a online game with bout 100 players online at one time a couple of years ago... it was still back in GM 8 and GMS 1.X days. I know bout the system, though I did not implement it the best way as the server did not know bout the physics and speeds and jump heights in the game. It just redirected messages between clients and took the clients messages as correct :p So, my main concern is if its a good idea to have the server run the actual game to have correct speed and estimation of player movement.

For the packet-loss: It seems there is a way to enable a "reliable" UDP mode in GMS 2.X... I think it takes care of making sure messages are delivered... the order can still be wrong though
 

The-any-Key

Member
If you want around 100 players now. I suggest you continue use a dedicated server. Tips in another language ex node. You can however make the game server in gml to handle physics and let the node server deliver the updates to the clients.

But if you plan a small 8 player game you can keep all in the same app.
 
B

BlaXun

Guest
Wait what? That does sound interesting. So I do have a server in GML... it simulates all clients... know bout the world and thus can get the "correct" location and such... meaning it has a direct "connection" using UDP.... where in this constellation does that Node server come in? Would that server communicate with the node server and the node server back to all clients? Or are clients not actually connected to the GML server but to node and the node server communicates with the GML server... so the GML server is never reachable "from the outside".
 

The-any-Key

Member
so the GML server is never reachable "from the outside".
In theory yes. You can have a couple of game servers in game maker that handle a specific area. The node server could handle the communication between server<>client more efficient. Because the node server could bundle the clients messages and send it as a packet to the correct game server. So the server can just send one update to the node server and the node server send it to the clients that need the info.
 
B

BlaXun

Guest
So it behaves like a cluster? You'd have "zone-servers" which handle just a couple of rooms... this would be good for performance. You have a "master server" for system-wide events.... and then you would have that Node server that is a relay of messages across servers and clients....

How would that additional structure of clustered servers behave in regards to delay? Did you ever attempt something like this? Its very interesting. Probably not needed for the game I plan... but its just fun thinking but it and I'd like to give such a clustered system a chance :)
 

The-any-Key

Member
I have tested game servers that run physics for one room. So each room for a own server. It works well in small tests.

Tested relay servers and it just adds 20-40ms to the delay.

But I have not yet tested to make these work together with a node server that bundle the communication. But you can mask delay with effects or sounds. Also the game design for a online game is to make the movement a bit slower. AAA companies have 1-2 seconds delay in their online games. The end game is to allow many players to be connected and have fun.
 
B

BlaXun

Guest
I’d still like to continue with this discussion. I think it would be great if client and server shared the same codebase. However, imagine a game with hundreds of smaller rooms for players. Right now GMS can only have one active room loaded at a time. What would be a good approach to solve this?

With such small rooms I think it would be a waste to have one server for each room.

It might be best developing a engine where you could load the rooms from a serializable format (come to think of it... isn’t GMS 2.2.0 capable of doing this)... it could then load multiple maps into a single room perhaps....

Do you have a nice idea how to solve this? I don’t want to go down a wrong path if I should create a new online game... in my previous games the server never knew bout the maps and just believed whatever the client said.
 
B

BlaXun

Guest
I have tested game servers that run physics for one room. So each room for a own server. It works well in small tests.
How many instances of servers (GMS executables) did u have running at the same time in such a scenario? I mean they all have a UI and would eat up memory. They would take time to start and setup correctly... While it sounds good in theory i doubt it would work for a game with many many smaller maps
 

The-any-Key

Member
How many instances of servers (GMS executables) did u have running at the same time in such a scenario? I mean they all have a UI and would eat up memory. They would take time to start and setup correctly... While it sounds good in theory i doubt it would work for a game with many many smaller maps
In my tests I only setup one main host on my computer that controlled a room. While some friends joined and was used as "acting" servers for one room each. So it was around 4 computers.

If I would go on a bigger scale I would remove the UI and have dedicated game servers and have the game designed for this type of architecture. So the game would have bigger rooms.
The main idea was not to host one server per room. Only to host servers for important areas, so they would be more secure.
Ex if a game had 150 areas. I would not setup game servers for all. I would bundle the important areas into 3-5 areas and run my own game servers without any UI.
For the rest, I would make players, that is currently in the area, as "acting" servers for the area, while they are in it. The player himself would not know, if he was an "acting" server or not. To make things a little more secure.
 
B

BlaXun

Guest
In my tests I only setup one main host on my computer that controlled a room. While some friends joined and was used as "acting" servers for one room each. So it was around 4 computers.

If I would go on a bigger scale I would remove the UI and have dedicated game servers and have the game designed for this type of architecture. So the game would have bigger rooms.
The main idea was not to host one server per room. Only to host servers for important areas, so they would be more secure.
Ex if a game had 150 areas. I would not setup game servers for all. I would bundle the important areas into 3-5 areas and run my own game servers without any UI.
For the rest, I would make players, that is currently in the area, as "acting" servers for the area, while they are in it. The player himself would not know, if he was an "acting" server or not. To make things a little more secure.
Yeah, removing the UI is something I would do too. I hope one dedicated server is enough for starters. However, I want to have the system flexible enough to be able to host "zone servers" on different dedicated servers.

My plan, which I want to prototype sometime soon, consists of the following, but can probably be simplified

  • A login/register server
  • A relay and database server
  • A unspecified amount of zone servers

The client and zone servers are GML and share the same codebase. This will allow for exact simulation of game mechanics.
The login/register server and the relay+database server will not be written in GML later on, but for fast prototyping I will use GML.

The login/register server will allow the clients to register a account and login.
This server will then inform the relay server about a authentication.
The relay server is the only instance that has access to some sort of database (and thus is pretty modular in the grand scheme and can be replaced later on).
The login server will create some kind of authentication token on login, send it to both client and relay server and then the client will be redirected to the relay server.
The relay server gets all relevant information from the database and redirect the client to one of the multiple zone servers. The zone server is notified about the incoming connection with the generated token.
The zone server (hopefully) accepts the incoming connection and the game can now truly start.

Well, thats the idea behind it...
For the zone server I have the plan that it consists of one big room only... no idea how big that room could be in the end... I won't need any UI at all on server in the end, but the client will have the same big room in which all the gameplay happens... no idea how intelligent GMS is in deactivating graphics outside of the visible range of the player... will need to do some testing regarding this. Anyway, a zone server will cover a big part of the game... and I hope to let the relay server be able to boot up new instances of a zone server in case it reached its capacity limit... and thus I would have some kind of "channel-system".

I'll try to do this all using UDP and I am excited to see how it will turn out in the end. Oh, I don't plan to implement some kind of delta-timing to smoothen gameplay... no idea how this will turn out, but I simply expect the clients computer to be powerful enough to run a GMS game with a minimalistic-resolution at a good framerate. I fear I am already over-engineering all this, but thats part of the fun for me I think :)
 
B

BlaXun

Guest
So I just stumbled upon this: https://forum.yoyogames.com/index.php?threads/headless-support-for-game-maker-studio-2.26454/

It seems running a server in GMS is a bad idea. I mean, yeah, I knew its not ideal... and if I thought I got the skills to simulate all the physics and timing in another language I would probably give it a try, but having all maps redundant on the client and server, re-writing all the GMS internal calculation for collision and physics and data structures, alarms and whatever sounds like just so much overhead that I'd argue GMS isn't even meant for realtime multiplayer experiences as soon as you want to have more then just a handful of players. I'd like to throw this topic into this discussion too if you guys feel like it.

Edit:
This is what I had in mind for the login process

 
Last edited by a moderator:
B

BlaXun

Guest
I realize the GMS Step-System will be at a limit somewhere down the line. To prevent this the use of a non-GML relay-server was introduced if I am correct.
So, a (for ex) Node server could process all the messages without a limit, stores packets for a short (!) amount of time, redirects it to the GML server, the GML server does the physics simulation, sends it back to the relay (Node) server and that Node server sends it back immediatly to the client?

I am having a hard time imagining all this.
@The-any-Key , can you elaborate further what the idea behind the non-GML server is? I think I know the reasoning behind it, but I just want to make sure.
It sounds a little annoying to switch to UDP to get decreased delay, but then we introduce another server that is non-GML which will just add delay due to more transmissions between that non-GML server and the actual GML-server.
I'd really like to get this discussion going again and I'd love to share my results as soon as I actually get the time to develop something

Edit:
This is how I understand it
 
Last edited by a moderator:

Mert

Member
Also, notice that Async event is not actually "async", and gets triggered like STEP event. It has to wait for the last step run to finish itself. In 60 room speed environment, it's fine. No problem.

Secondly, I tried that relay thingy. Years ago, my friend made a server with boost library(https://www.boost.org/). We set up the whole thing as:

We had two applications, one written in C++ and the other with Game Maker Studio. Cpp server was responsible with getting packages from clients, send these packets to Game Maker server, get the new updates from Game Maker server and sends them back to clients at last.Getting packages from clients and processing them in Cpp Server was super fast! We also had Mysql database running so instead of getting informations from Game Maker server, we get these from a Mysql database.

An example process is like:
  1. Client sends a connection request to Cpp server. Handshakes go on and we get connected finally.
  2. Server forwards this message to Game Maker Server. GM server creates a player object. Places this player randomly on a place in room. Sends the coordinates back to Cpp server. This server sends this package to all clients including the origin-of-message client.
  3. Game Maker Server creates random zombies around the map. Creates 8 and disables them. Sends this info to Cpp server. Cpp server sends this message to all clients as well. We also waited for all players to get this message. After confirmation, we inform Game Maker server to activate zombies. These zombies had predictable movement mechanism so that we didn't send each of their coordinates to players. But once a player made contact with a zombie, then this zombie is extra-activated and it's coordinates and actions are updated periodically.
My observations :
  • The delay between Gms server and Cpp server was not too much. It's like using an API(practically it was).
  • Make sure to split the job and let Cpp server(or Java/Python/C#... server) to handle the most job. Use as many external resources as possible to reduce your dependency on Game Maker. We also tried replicating zombie actions on cpp server. I believe with Box2D library, you can cut the physics part from GM.
  • Using multi-threaded server with single threaded server is confusing. But we had solution for this too, running multiple GM servers! Basically we split threads over players and one for the master operations. For each player group, assign a GM server. The confusing part comes as you also have to sync multiple GM apps with each other, but don't worry. You'd only need to synchronize important informations.
It was years ago, don't have the files now and I pretty much forgot the technical details. I used my computer as a server, and also run one test on digitalocean.

Hey, btw I remember you from that Slime online games :D
Edit : grammar
 

The-any-Key

Member
It sounds a little annoying to switch to UDP to get decreased delay, but then we introduce another server that is non-GML which will just add delay due to more transmissions between that non-GML server and the actual GML-server.
I would use UDP for all the communication. The idea about the relay server is to bundle messages. The gml game server is too slow to handle it alone. The delay is unavoidable. You can however use the delay and bundle steps before you send a response back to the relay server. This would create a controlled delay and make gameplay more smooth.
 
B

BlaXun

Guest
@Mert Splendid, thats just the way I had imagined it to work.
I still have no idea what to use for the server. I'd like to use a swift server... but as swift is still pretty young I would probably be better off using a well established language... Node.JS comes to mind.... or even python.
I'm still not sure how exactly the packet-bundling on the non-GML relay-server should be done... I mean, what is the sweetspot between bundling em up and finally sending em. I hope to get the time soon to just give it a try. Unfortunately, for a realistic scenario I'd need a dedicated server... which I do not have right now. I might just be able to run a 2nd computer somewhere however...
 
B

BlaXun

Guest
I would use UDP for all the communication. The idea about the relay server is to bundle messages. The gml game server is too slow to handle it alone. The delay is unavoidable. You can however use the delay and bundle steps before you send a response back to the relay server. This would create a controlled delay and make gameplay more smooth.
Yeah, UDP is definetly the only protocol I WOULD use if I were to start prototyping this. I am not sure how good GML performs with UDP. Did you ever try that "Reliable UDP"-Setting (network_config_enable_reliable_udp)?
How safe is it to use that? I did work with TCP before.... for quite some time actually. But I have no idea how much work needs to be done on making sure UDP packets actually reach the server and are in order. I mean, the reliable mode should make sure the packets arrive... the order can still be garbage though, which is okay, but I would like to know how clever that yoyogames "reliable udp" mode is.... or if I am better of implementing that part myself
 

The-any-Key

Member
Game maker use LE buffers. And if you use node. You need to manually add a null pointer when you write strings to a buffer that you want GM to be able to read it. The stings need to be in utf8.
 
B

BlaXun

Guest
Game maker use LE buffers. And if you use node. You need to manually add a null pointer when you write strings to a buffer that you want GM to be able to read it. The stings need to be in utf8.
I am afraid I don't know what a "LE Buffer" is ...and a simple google search didn't help either. What language would you personally recommend to use for the server? I was also told that eNet (http://enet.bespin.org/) might be a good way to do this. Unfortunately outside of GML my skills are more with SWIFT, Objective C, Java, some .NET, little python. I am struggling to decide here :) Thank you for all your valuable information already
 
B

BlaXun

Guest
Never tested.
I use a id system to track lost packets and resend them manually.
I'm interested in this. I imagine it like this:

Each packet gets a Id assigned. Each packet sends its Id AND the Id of the previous packet. When the server gets that packet and the packet with the "previousPacketId" was not received yet the server requests it from the client again.

however, how would the client know the packet was received? Will the server INFORM the client actively about every received packet? I mean, the client MUST be able to store all packets for a while before discarding em. How did you handle this? Very valuable information in this topic :)
 

The-any-Key

Member
B

BlaXun

Guest
I use nodejs.


Little endian. It's actually how you write and read from the buffer.
Ex
https://nodejs.org/docs/latest/api/buffer.html#buffer_buf_readint16le_offset


You only need to send one Id per packet and check if the id was expected.
1 ok
2 was lost
3 where is 2? Request 2 again. Store 3 until 2 is received.

Each side store the last 500 sent buffers.
That sounds reasonable.... the client sent packets immediately? Or did it bundle up packets for... a few steps?
 

The-any-Key

Member
I bundle a few steps before I send it. It depends on how many players I want in each relay. Less sends mean more players per relay.

You can also compress data when you bundle.
 
B

BlaXun

Guest
I bundle a few steps before I send it. It depends on how many players I want in each relay. Less sends mean more players per relay.

You can also compress data when you bundle.
Intelligent compression could go a long way... this would be great to minimize delay further. What kinds of compression did you use?
 
B

BlaXun

Guest
Hey there, I am still alive.
Does anybody have an example on communication between GMS 2.X and either a Python server or NodeJS server?
With python I can send strings just fine, but I can’t get numerics sent/receives correctly... probably some kind of encoding problem.
Thx in advance
 

The-any-Key

Member
You probably use the wrong endian,
Node example:
Code:
server.on('message', (msg, info) =>
{      
    // Check if baned
    if (sf.CheckIfBanned(info)==false)
    {
        // Show info about message
        if (showDebug) console.log('\nData received from client : ' + JSON.stringify(msg));
        if (showDebug) console.log('Received %d bytes from %s:%d',msg.length, info.address, info.port);
      
        var b=0;
        // Read buffer
        var msg_type=msg.readUInt8(b); b+=1;
        // Show message type
        if (showDebug) console.log('type : ' + msg_type);
      
        // Handle messages
        switch(msg_type)
        {
            case type_player_update:
                // Update player
                var player_index=msg.readUInt16LE(b); b+=2;
                var player_public_token=msg.readUInt16LE(b); b+=2;
                setImmediate(sf.UpdatePlayer,msg,b,info,player_index,player_public_token,b,false);
                break;
...
 
Last edited:
B

BlaXun

Guest
So NodeJS has no trouble with the format GMS uses for numeric types?
With python I could not read numerics... I got gibberish instead
 
B

BlaXun

Guest
You probably use the wrong endian,
Node example:
Code:
server.on('message', (msg, info) =>
{
    // Check if baned
    if (sf.CheckIfBanned(info)==false)
    {
        // Show info about message
        if (showDebug) console.log('\nData received from client : ' + JSON.stringify(msg));
        if (showDebug) console.log('Received %d bytes from %s:%d',msg.length, info.address, info.port);
 
        var b=0;
        // Read buffer
        var msg_type=msg.readUInt8(b); b+=1;
        // Show message type
        if (showDebug) console.log('type : ' + msg_type);
 
        // Handle messages
        switch(msg_type)
        {
            case type_player_update:
                // Update player
                var player_index=msg.readUInt16LE(b); b+=2;
                var player_public_token=msg.readUInt16LE(b); b+=2;
                setImmediate(sf.UpdatePlayer,msg,b,info,player_index,player_public_token,b,false);
                break;
...
Could I explain what that “b+1” and “b+2” is all about? Is it pointer shifting in a buffer or something? Thanks in advance

Edit:
Okay, got it. It’s the offset after reading a value depending on that values type.

Edit:
I still didn’t figure out how to read strings though.... will I have to add a byte to let the node server know the length of the string so I get the size correctly?

Edit:
I can read bools and several numerics. I still got problems with floats though. This is my code

GML
Code:
socket = network_create_socket(network_socket_udp)

buffer = buffer_create(1, buffer_grow,1)

buffer_seek(buffer,buffer_seek_start,0)

buffer_write(buffer,buffer_bool,true) // 1 byte
buffer_write(buffer,buffer_bool,false) // 1 byte
buffer_write(buffer,buffer_u8,1) // 1 byte
buffer_write(buffer,buffer_u16,2) //    2 byte
buffer_write(buffer,buffer_u32,3) // 4 byte
buffer_write(buffer,buffer_s8,4) //1 byte
buffer_write(buffer,buffer_s16,5) //2 byte
buffer_write(buffer,buffer_s32,6) //4 byte
//buffer_write(buffer,buffer_f16,1.0) //2 byte (NOT SUPPORTED!?)
buffer_write(buffer,buffer_f32,2.0) //4 byte
buffer_write(buffer,buffer_f64,3.0) //8 byte
//buffer_write(buffer,buffer_string,"String") //Any size, null terminating
//buffer_write(buffer,buffer_text,"Text") //Any size, no null terminating

network_send_udp_raw(socket,SERVER_IP,SERVER_PORT, buffer, buffer_tell(buffer))
NODE SERVER
Code:
var PORT = 33333;
var HOST = '127.0.0.1';

var dgram = require('dgram');
var server = dgram.createSocket('udp4');

server.on('listening', function () {
    var address = server.address();
    console.log('UDP Server listening on ' + address.address + ":" + address.port);
});

server.on('message', function (message, remote) {

    var b = 0;
    var boolTrue = message.readUInt8(b); b+=1;
    var boolFalse = message.readUInt8(b); b+=1;
    var uIntEight = message.readUInt8(b); b+=1;
    var uIntSixteen = message.readUInt16LE(b); b+=2;
    var uIntThirtyTwo = message.readUInt32LE(b); b+=4;
    var sIntEight = message.readInt8(b); b+=1;
    var sIntSixteen = message.readInt16LE(b); b+=2;
    var sIntThirtyTwo = message.readInt32LE(b); b+=4;
//    var floatSixteen = message.readFloatLE(b);b+=2;
    var floatThirtyTwo = message.readFloatLE(b);b+=4;
    var floatSixtyFour = message.readFloatLE(b);b+=8;
//    var string = message.
//    var text =


    console.log(JSON.stringify(message));
    console.log('boolean true: ' + boolTrue);
    console.log('boolean false: ' + boolFalse);
    console.log('uInt8: ' + uIntEight);
    console.log('uInt16: ' + uIntSixteen);
    console.log('uInt32: ' + uIntThirtyTwo);
    console.log('sInt8: ' + sIntEight);
    console.log('sInt16: ' + sIntSixteen);
    console.log('uInt32: ' + sIntThirtyTwo);
//    console.log('float16: ' + floatSixteen);
    console.log('float32: ' + floatThirtyTwo);
    console.log('float64: ' + floatSixtyFour);
});

server.bind(PORT, HOST);
LOG OUTPUT FROM NODE SERVER
Code:
{"type":"Buffer","data":[1,0,1,2,0,3,0,0,0,4,5,0,6,0,0,0,0,0,0,64,0,0,0,0,0,0,8,64]}

boolean true: 1

boolean false: 0

uInt8: 1

uInt16: 2

uInt32: 3

sInt8: 4

sInt16: 5

uInt32: 6

float32: 8.96831017167883e-44

float64: 0
As you can see I can't handle floats yet. And the GMS 2.X manual states that float16 isn't supported yet. I also need a way to figure out how long a string will be when sent by HTML. I might be able to just search for the delimiter in the buffer though... will check it out later.
I'm glad I am finally making some progress :) Thanks again everybody. I'd like to continue discussing here :)

Edit:

Okay, getting the string from the buffer was fairly easy
I hope I am correct in assuming that 0 is the delimiter char

Code:
var delimiterPos = message.indexOf(0,b)
var stringi = message.toString('utf8',b,b+delimiterPos)
console.log(stringi);

Edit:
Got floats done too. I got everything I need now. Next up I'll try to think of a good way to actually handle packets. I'll keep you updated on my little experiments. Maybe this will be of help to somebody
 
Last edited by a moderator:
B

BlaXun

Guest
Game maker use LE buffers. And if you use node. You need to manually add a null pointer when you write strings to a buffer that you want GM to be able to read it. The stings need to be in utf8.
This is the way I try it ->

Code:
var client = dgram.createSocket('udp4');
    var clientBuffer = Buffer.alloc(1024);
    var pos = 0;
    pos += clientBuffer.write("This is a test" + "\0", pos);
    pos += clientBuffer.write("Another test" + "\0", pos);
    pos = clientBuffer.writeUInt8(true,pos);
    pos = clientBuffer.writeUInt8(false,pos);
    pos = clientBuffer.writeUInt8(255,pos);
    pos += clientBuffer.write("The end..." + "\0",pos);

    clientBuffer = clientBuffer.slice(0,pos);
    console.log(JSON.stringify(clientBuffer));
 
    client.send(clientBuffer, 0, clientBuffer.length, remote.port, remote.address, function(err, bytes) {

        if (err)
            throw err;
     
        console.log('UDP message sent to ' + HOST +':'+ PORT);
        client.close();
    });
In GM I use a message_show to get the following printed

"This is a test".

I guess I am doing something wrong bout the null pointer you mentioned. Do you think you could let me in on your secret? ;)

Edit:
Okay, I thought I'd have to append a null pointer after each string... which was wrong. Only one null pointer at the end of the message is needed.

Edit:
As far as I can tell I can not just simply use the buffer_read(... function from within GML with the buffer I get from Node. I can read all strings at once, all uInt8 at once... but I would like to find a nice way to read from the buffers I get sent by Node. How do you guys solve this?

Edit:
Okay, full-stop!
It actually is correct to add a null pointer after every string. Once I did this I was able to read everything correctly from the buffer. I am doing it like this:

Code:
var client = dgram.createSocket('udp4');
    var clientBuffer = Buffer.alloc(1024);
 
    var pos = 0;
    pos += clientBuffer.write("This is a test", pos);
    pos = clientBuffer.writeUInt8(0x00,pos);
    pos += clientBuffer.write("Another test", pos);
    pos = clientBuffer.writeUInt8(0x00,pos);
    pos = clientBuffer.writeUInt8(true,pos);

    pos = clientBuffer.writeUInt8(false,pos);
    pos = clientBuffer.writeUInt8(255,pos);
    pos += clientBuffer.write("The end..." + "\0",pos);
    pos = clientBuffer.writeUInt8(0x00,pos);
    clientBuffer = clientBuffer.slice(0,pos);
 
    console.log(JSON.stringify(clientBuffer));
    client.send(clientBuffer, 0, clientBuffer.length, remote.port, remote.address, function(err, bytes) {

        if (err)
            throw err;
        
        console.log('UDP message sent to ' + HOST +':'+ PORT);
        client.close();
    });
I still have to manage to write uInt16, uInt32, sInt8, sInt16, sInt32, Double and Float .... and read em on GML. Somehow I got weird buffer sizes the last few tries.

Edit:
I can read all types in GML now... only float and double are missing. I am trying like this:
pos = clientBuffer.writeFloatLE(1.123456,pos);
pos = clientBuffer.writeDoubleLE(1234.567891011121,pos);

In GML I get 0.0 for both values.

Edit:
Okay, I am stupid. I ignored the limits of a 32bit Float and 64bit Float it seems. Woopsie
 
Last edited by a moderator:
B

BlaXun

Guest
In theory yes. You can have a couple of game servers in game maker that handle a specific area. The node server could handle the communication between server<>client more efficient. Because the node server could bundle the clients messages and send it as a packet to the correct game server. So the server can just send one update to the node server and the node server send it to the clients that need the info.
Do you have a recommended size on how big the buffer should be at maximum before sending it from relay server to node server? Also, do u have a recommendation on how long to wait on queueing packets before sending em to GM?
 

The-any-Key

Member
Do you have a recommended size on how big the buffer should be at maximum before sending it from relay server to node server? Also, do u have a recommendation on how long to wait on queueing packets before sending em to GM?
I send around 700 bytes per UDP packet. But this is because of the MTU limit. I would use a private network to communicate between the Relay and Communication server. This would allow bigger packets, less lost packets and more speed. Because the relay would handle the queue, I would make the communication server send the packet directly to the game server whenever it comes.

How long to wait until the relay server send to the communication server is hard to answer. It depends on game mechanics, how many players you want, players actions... You also need to time actions and movement in the game so it fits with the delay between the message updates from the game server. The most important is to keep all values configurable. So you can easy change them.
 

Pfap

Member
There is a Node framework for Gamemaker on GitHub called Patchwire
https://github.com/twisterghost/patchwire I just started playing around with it, but it seems pretty good. I'm not much of a javaScript expert, it seems like a pretty bloated language though. I have a hard time "thinking" in javaScript, but can read it for the most part, so I'm not really sure what problems to expect with the above framework. One thing that does not make perfect sense to me is the use of multiple "ClientManagers" it seems to me like a memory leak waiting to happen...
Maybe, I just don't know what I'm doing in javaScript; although, I assume that when a reference to an instance is lost that the garbage collector removes the instance.



If you are familiar with objective c, it might be a better approach to find a c or c++ solution? I've heard that objective c is pretty similar to c.
Still, the only reason I chose javaScript was to get the Node v8 runtime which is a c++ program. This whole programming thing would be easier if there was a clear choice, but there are way too many to choose from and so, it is complex.


Edit:
It is also a tcp and JSON solution... just reviewed the postings above me.
 
B

BlaXun

Guest
I am now at a point where I am collecting packets to relay em all at once to the GM server. Up until now I relayed the packets directly, I am now using a interval of about 100ms (will change this). So, for up to 100ms I am collecting packets before sending em. However, Can I simply send all those packets at the same time when that interval is done, or should I create one "big" buffer and bundle up all packets? With the later method I would have less messages sent, but I would have to add some kind of header to the packet so the GM client knows at what offset what packets starts.

I think creating some bundled buffers (each about 1024 byte maybe) would be good for performance. What do you think about this?

Sidenote:
For a small project right now I am able to use a iPhone to control a character in a GM Client ... using nodeJS as a relay server. This is quite fun :)

Edit:
That header with offset values isn't even needed I think... still, I'd like to know if creating bigger packets to bundle smaller packets would help performance. Did anybody test this yet?

Edit:
Right now I decided to simply send packets every 10ms. As long as those 10ms didn't expire I write incoming packets to one "big" buffer. As soon as the 10ms elapse I relay thas "big" packet to the GM server to handle it all. It works... delay is okay. I could improve this further. I think as long as the server can keep 60FPS stable everything should be fine, right? And having many clients connected to it at once would quickly exceed the 60 FPS limit (at least if I sent each step, which I am not doing... I am doing server-side-prediction first)
 
Last edited by a moderator:

The-any-Key

Member
or should I create one "big" buffer and bundle up all packets? With the later method I would have less messages sent
Bundle to big packet.

I think creating some bundled buffers (each about 1024 byte maybe) would be good for performance. What do you think about this?
I would use 700 bytes per send because of the MTU limit. I have tested with 800-1500 bytes buffers and they can be blocked when you go international.
 
B

BlaXun

Guest
Thanks a lot. I actually just managed to bundle packets and relay them.
However, there is one problem I am running into right now.

My server architecture is server-side-prediction with only sending key-strokes and releases.
I do send packets every 10 ms.

However, if the player would quickly tap the left or right key he would move on his local client, but because the server gets both packets at once it won't do any movement at all.
I think the answer to this is that sending strokes and releases only won't actually work and I should instead provide a steady stream of latest client coordinates.
However, I am very afraid the server would not be able to handle this very quickly.
 

The-any-Key

Member
but because the server gets both packets at once it won't do any movement at all.
Then you need to rethink your architecture. Sending keystrokes like this often result in bad sync. Collect all keystrokes on client each step:
1 0 1 (1= key down 0 = key up)
Bundle in a send.
On remote put the keystrokes in a queue and "play" the each keystroke in a own step:
1 0 1

If you get 2 or more packets delivered at the same time, just queue them after each other. This would make so you dont miss a single keystroke.
 
B

BlaXun

Guest
Hm, okay. Sure, that would work, but how would the server simulate jumping for example? I mean, the server gets a jump-packet. Since the server knows the physics it initiated the jump. Due to the delay the key-states will get... well, delayed but the jump physics are already in action... how would I deal with such a scenario? How would I inform the player about his correct position? The benefit of sharing the client and server in the same GM project is that I have shared physics and thus can easily simulate those. If I would now „disable“ the physics to do all the simulating step-by-step I would ruin the purpose of a shared project. To me it seems as if I am misunderstanding some key-concept in all this (it’s still great fun so far). Thanks for reading. I’ll be waiting for your feedback :)
 

The-any-Key

Member
You playback the keypresses step by step on the server. So you will have a queue with future keypresses on the server. The queue need to be the size of the delay. Before the server run out of keypresses. New keypresses should have arrived from the client.
 
B

BlaXun

Guest
You playback the keypresses step by step on the server. So you will have a queue with future keypresses on the server. The queue need to be the size of the delay. Before the server run out of keypresses. New keypresses should have arrived from the client.
This makes sense, I am totally with you there. However it also sounds very unstable in regards to processing that traffic. This basicly means every step event is 100% filled with what to do... if even a single packet would be received outside of that process in GM then delay would hit in, would it not?
 

The-any-Key

Member
This makes sense, I am totally with you there. However it also sounds very unstable in regards to processing that traffic. This basicly means every step event is 100% filled with what to do... if even a single packet would be received outside of that process in GM then delay would hit in, would it not?
I use a regulator. Whenever the queue is to small (the queue is empty). It will hold/pause some milliseconds so it can queue more from the server. Whenever the queue is too big, it will consume some extra from the queue in the same step. But you need to design the game so you can object vise hold/pause or consume extra keys. This is why I use state sync instead of key sync. State sync is more forgiving with wait and consume extra data.
 
B

BlaXun

Guest
Okay... yeah, this all sounds fine...
I still didn't find the time to change my architecture.

Another thing I'd like to talk about is delay. Sure, we use a relay server, we bundle packets on client, relayserver and gm-server... which will create delay. However, what happens if a UDP packet isn't received? I mean, as far as I understood the endpoint will request that packet again as soon as it gets another packet that relies on the previous packet (sure, I have to write that myself)... but this will introduce more delay... and since its step-based at that point it means that particular client would never recover from delay again, or would it not?

Edit:
There is another, very basic, question I have. In regards to sending and receiving packets.
How should UDP packets be handled when it comes to losses? Right now I simply just don't care if packets are not received... but I know I have to change this... its just that in my prototype this isn't really a problem. So yeah, the client sends a packet... should the server send a ACK back that it received the message... or should the server simply request the packet again if it gets a packet with a higher id?

Perfect way:
Client sends packet with Id 1
Server sends ACK with packet Id 1 back

Possible solution 1:
Client sends packet with Id 1
Server didn't receive packet
Client sends packet with Id 2
Server now requests packet with Id 1 again
Client resends packet 1 (that is... if it got that packet that requested packet 1) :D

Possible solution 2:
Client sends packet with Id 1
Server didn't receive the packet and thus didnt reply with a ACK
Client notices that it didnt get a ACK and resends that packet
Server gets the packet and is happy

For solution 2: If the packet was received twice and the ACK just got lost on its way to the client then the server will just ignore that packet.

So, I think solution 2 is the better way... what I don't like about all this resending is that this requested packet will probably still have to go through that packet bundling ...so it would be delayed even more... or I could send it directly... I mean, it would be sent to the nodeServer ....and the node server should be able to handle this.
 
Last edited by a moderator:

The-any-Key

Member
and since its step-based at that point it means that particular client would never recover from delay again, or would it not?
If a packet is delayed you pause and continue when data is received again (and make your queue bigger to avoid it in the future play). You then regulate the data so when you receive to many packets you consume more packets to keep the players in sync.

How should UDP packets be handled when it comes to losses? Right now I simply just don't care if packets are not received... but I know I have to change this... its just that in my prototype this isn't really a problem. So yeah, the client sends a packet... should the server send a ACK back that it received the message... or should the server simply request the packet again if it gets a packet with a higher id?
I dont use ACK because this makes the server send extra packets that could be used to host more players instead. But it is up to you and how the game mechanics works. There is no perfect situation here. Packets will be lost. If you should re-send them? It is up to you. The idea is to make the game look good for the players. If you can make the game look good with some missed packets then go for it.

Possible solution 1:
Client sends packet with Id 1
Server didn't receive packet
Client sends packet with Id 2
[Server notice the wrong id]
Server now requests packet with Id 1 again
Client resends packet 1 (that is... if it got that packet that requested packet 1)
I use this in smaller games. The server can handle the extra requests. But if you go for a 100 player game. You dont want the server to request packets and overload itself. It might be that the client it is trying to reach is out of reach. I would instead wait for the next client update.

So, I think solution 2 is the better way... what I don't like about all this resending is that this requested packet will probably still have to go through that packet bundling ...so it would be delayed even more... or I could send it directly... I mean, it would be sent to the nodeServer ....and the node server should be able to handle this.
My rule in small games (4-16 players): Server send and request missing packets. Client send and request missing packets.
My rule in bigger games (>50 players): Server send packets. Client send and request missing packets.

Whenever the client request a missing packet I use a private line between the server and the client. So the server will just send the packets I requested. Even if it not update time.
Too keep CPU low on the server I would in some cases use a second server to handle clients resend requests. To make sure the workflow on the main server is low.
 
B

BlaXun

Guest
If a packet is delayed you pause and continue when data is received again (and make your queue bigger to avoid it in the future play). You then regulate the data so when you receive to many packets you consume more packets to keep the players in sync.
I am not quite sure I understand? Does this mean the packet-queue for that one player will expand? So in theory it can always expand but never shrink again? So, the longer the session the higher the risk of a big queue for that connected client?

I dont use ACK because this makes the server send extra packets that could be used to host more players instead. But it is up to you and how the game mechanics works. There is no perfect situation here. Packets will be lost. If you should re-send them? It is up to you. The idea is to make the game look good for the players. If you can make the game look good with some missed packets then go for it.
Hm, yeah, makes sense. However, some packets are more important than others. I think movement can be discarded in some situations... and maybe the client could send his absolute position from time to time... or how do you sync client-side and server-side positions between clients?

I use this in smaller games. The server can handle the extra requests. But if you go for a 100 player game. You dont want the server to request packets and overload itself. It might be that the client it is trying to reach is out of reach. I would instead wait for the next client update.
My goal actually WOULD BE a game with > 100 players... probably not next to each other (within the game world) so there would be many packets... but still... more then just a few :)
Overloading would be a risk, I agree. It probably depends on the packet type.


My rule in small games (4-16 players): Server send and request missing packets. Client send and request missing packets.
My rule in bigger games (>50 players): Server send packets. Client send and request missing packets.
Okay, this sounds good. So if the server is missing some packets it will just continue as usual. However if the client is missing stuff from the server (which we would consider important!) the client requests these again...
Maybe not all packets are important for that client though... maybe some other client movement-packets aren't even as important ... but this would cause desync... maybe this could be masked in the following:

Lets assume the client runs at 60fps... so 60 steps a second.... which means 60 packets a second. However, we send (bundled) packets every 0.5 seconds. So we have 30 packets of key-states we send to the server. The first (and?) last packet in that sequence could however be a absolute x,y for that client so we correct the position before starting the movement. This sounds like a good way to me how to mask missing movement packets.

Whenever the client request a missing packet I use a private line between the server and the client. So the server will just send the packets I requested. Even if it not update time.
Too keep CPU low on the server I would in some cases use a second server to handle clients resend requests. To make sure the workflow on the main server is low.
Yeah, I setup my server like this. I can send packets directly without waiting for the queue and then there is the global queue that sends packets to the clients. A second server is my plan as well, but not for the same map actually... or wait... this could be a good way. Maybe I could setup the server so it starts new instances of the relay server when needed.

In the end I think movement packets should be considered discardable... but some packets are sensitive and they should reach the client (and even server).
When we now send such a sensitive packet to the server and the server doesn't react for a "longer" period I could just send that one packet again from the client...
When the server sends to the client and the client is missing a previous packet he could request that.

All in all I think SOME packets could be marked as IMPORTANT and these should work with a ACK. This would only be a few packets such as "use an item", "change a map" and such.
Once again, thanks for the productive discussion. It's been a long time since I enjoyed a conversation bout technical stuff this much
 

The-any-Key

Member
I am not quite sure I understand? Does this mean the packet-queue for that one player will expand? So in theory it can always expand but never shrink again? So, the longer the session the higher the risk of a big queue for that connected client?
You expand when needed, and you shrink when needed. You try keep the queue in a stable state.

Hm, yeah, makes sense. However, some packets are more important than others. I think movement can be discarded in some situations... and maybe the client could send his absolute position from time to time... or how do you sync client-side and server-side positions between clients?
In these situations I actually make the server response with a byte or make the server continue send other data, that was planned. Ex if the client send a init packet with some data and then we will request the map data after the server got the init data, I could make the server send a "init packet received" back to the client and then the client request the map data. But I could also just continue the logic. And after the server receive the init packet, it start send the map data to the client as a "init packet received, now here is your map data". In these cases we can skip the ACK message.

Maybe I could setup the server so it starts new instances of the relay server when needed.
I plan to use digitalocean for this (promo link: https://m.do.co/c/60302d933ffb)
You can tell nodejs to do shell commands, and using the digitalocean API, you can tell to create new VM and start them as servers when you exceed the player max. If you player base drops, you can destroy the VM servers. This makes the architecture auto-scale-able. It is very interesting.

In the end I think movement packets should be considered discardable... but some packets are sensitive and they should reach the client (and even server).
Some messages is always more important. Best is to always try make the client take the heat however and spare the server.
 
B

BlaXun

Guest
You expand when needed, and you shrink when needed. You try keep the queue in a stable state.
So I expand when I notice a packet is missing and I reduce as soon as possible when a sequence is complete and just got sent?

In these situations I actually make the server response with a byte or make the server continue send other data, that was planned
My idea for this (to reduce packet size) is to use a UInt16 for the initial packet identifier.

So I could use a range for identifiers. Let’s say I reserve 0 - 4000 for “unimportant” packets and 4001 - 8000 for important packets. This way the server and client both know what packets need an ACK and what don’t. For packets that need a ACK I append a UInt8 with a number that will be used to identify the missing packet.

You can tell nodejs to do shell commands, and using the digitalocean API, you can tell to create new VM and start them as servers when you exceed the player max. If you player base drops, you can destroy the VM servers. This makes the architecture auto-scale-able. It is very interesting.
Yeah, I’ll get to this at some point I think :) for now I will push it to my backlog.

Some messages is always more important. Best is to always try make the client take the heat however and spare the server.
Great sentence. “Always try to make the client take the heat - spare the server”. I’lol write this down as one of my guidelines for client-Server architecture.
 

The-any-Key

Member
So I expand when I notice a packet is missing and I reduce as soon as possible when a sequence is complete and just got sent?
This is on the client side. When bundled packets are received from the server. You expand the queue when you notice you are running out of steps to process locally. And when you notice the queue got a bunch of packets you need to process them faster to make the client keep up. This is done per client base. So each client can have a smooth gameplay. Whatever ping they might have.
 
B

BlaXun

Guest
This is on the client side. When bundled packets are received from the server. You expand the queue when you notice you are running out of steps to process locally. And when you notice the queue got a bunch of packets you need to process them faster to make the client keep up. This is done per client base. So each client can have a smooth gameplay. Whatever ping they might have.
Oooohhh...
I always thought this would be the case on both the client and server-side. I am struggling with how to run that queue faster. Since every packet is bound to one step I’d kinda have to combine packets I guess? So, run two packets in one step, but still in sequence... at least that’s how I would understand. But I think I’m getting into this more and more. My NodeJS server is also coming along pretty nicely. Thanks once more for your reply. Do you have any samples of your works? Any links or something? What kind of games are these?
 
Top