• Hello [name]! Thanks for joining the GMC. Before making any posts in the Tech Support forum, can we suggest you read the forum rules? These are simple guidelines that we ask you to follow so that you can get the best help possible for your issue.

Legacy GM [RESOLVED] Wrong packets in TCP connection between GM:S and Java

CodeManu

Member
Hi all!

After been struggling with this for about 30 hours by now I've decided to post it here because I don't think I can find an explanation by myself.

Introduction: I've been working on a Java server for a GM:S game, the connection is made through TCP. The Java server connects with GM:S without problem and start sending packets of 19 bytes. The packets are received by the GM:S client but sometimes the packet content is mixed with other packets. I know TCP is streamed so I write all the data received into a buffer and then when that buffer is 19 bytes long (at least) I extract the packet from there.

The problem: From the Java server I'm sending: u16 + byte + byte + byte + u16 + u16 + u16 + u16 + u16 + u16 + u16. So it's: 2+1+1+1+2+2+2+2+2+2+2 = 19bytes. When I read the data I'm receiving the 19 bytes packet should look like this: "25-4033380808080808080", but this packet is sometimes mixed as shown in the following image:
mixedpackets.png

I've highlighted in green the packets that are received fine, but you can see that the "white" packets are mixed.

The data is collected with the following code to avoid any changes from the data received. (This code runs in the Networking event)

buff = ds_map_find_value(async_load,"buffer");
buffer_seek(buff, buffer_seek_start, 0);
repeat(buffer_get_size(buff)) {
file_text_write_real(f,buffer_read(buff, buffer_s8));
}
file_text_writeln(f);

And if it helps, the data is sent from the Java server with the following code. (I know the code is ugly, I just wanted to make sure the data was sent and the packet size was fixed)

int xxx = input.readByte();
DataOutputStream os = ply.getOutputStream();
StreamConverter.buffer_write_u16(os, 55321);
os.writeByte((byte) 3);
os.writeByte((byte) xxx);
os.writeByte((byte) xxx);
StreamConverter.buffer_write_u16(os, 8);
StreamConverter.buffer_write_u16(os, 8);
StreamConverter.buffer_write_u16(os, 8);
StreamConverter.buffer_write_u16(os, 8);
StreamConverter.buffer_write_u16(os, 8);
StreamConverter.buffer_write_u16(os, 8);
StreamConverter.buffer_write_u16(os, 8);
os.flush();

My question is obviously: Why are the packets mixing their data? Shouldn't TCP protocol avoid this?

Thanks in advance guys, and let me know if you have any clues or need more info about this.

EDIT: The problem was that threads were sending their data but not in order, that caused the data to arrive mixed. Using a lock made them work as I wanted!
 
Last edited:

Mercerenies

Member
Hm... it might be handy to see your handshake code. That is, the code used to set up the connection on both ends. I suspect that the problem might be in how you're agreeing on the protocol at the very beginning.
 

CodeManu

Member
I'm not at home and can't post the code, but I believe there's no handshake, just raw connection between Java and Gm:S and the packets are sent raw too. I then add custom headers to the packets to be able to read them, but in the output log I've added to the post there's no data manipulation just the data received being added to the log file.
 

CodeManu

Member
You are sending unsigned 16 bits integers but reading signed 8 bits integers.
But that shouldn't be a problem, I'm reading byte by byte so I know I'll get the wrong values, but even if the values are wrong the packets should be always the same and shouldn't be mixed in some cases.
 

HammerOn

Member
I'm not understanding the image you attached.
The text file starts wrong (4033380808080808080 instead of 25-4033380808080808080) and the pattern you said is the right is not one the highlighted.
 

CodeManu

Member
I'm not understanding the image you attached.
The text file starts wrong (4033380808080808080 instead of 25-4033380808080808080) and the pattern you said is the right is not one the highlighted.
It's because the data in the image is shifted to make it easier to highlight. Here's one test log of this issue (I'm not sure if it is the same one of the image, but the same problem occurs): https://dl.dropboxusercontent.com/u/79218570/lg6.log
 

HammerOn

Member
Now it makes sense.
I don't know much about Java but seems like you aren't sending packages of 19 bytes but streaming it.
In the log you can see that sometimes a piece of the data is written, then comes a new line (file_text_writeln(f)), then the other piece of the data.
 

CodeManu

Member
Now it makes sense.
I don't know much about Java but seems like you aren't sending packages of 19 bytes but streaming it.
In the log you can see that sometimes a piece of the data is written, then comes a new line (file_text_writeln(f)), then the other piece of the data.
TCP is streamed indeed but that shouldn't affect the output of the log, even if the data received is streamed it shouldn't be mixed.
 

HammerOn

Member
But the log shows it is. Sometimes the byte 25 is received then a another complete 19 bytes is received then the rest of the previous buffer.

It's usually what happens when we use threads without locks.
Or two streaming to the same client.
 

CodeManu

Member
But the log shows it is. Sometimes the byte 25 is received then a another complete 19 bytes is received then the rest of the previous buffer.

It's usually what happens when we use threads without locks.
Or two streaming to the same client.
That might actually be what's happening, I'm don't have much experience with locks but you say that what's happening is something like:

Thread 1 -> starts executing
Thread 2 -> starts executing
Thread 2 -> finish executing
Thread 1 -> finish executing

That would explain the mixing data, is this possible to happen?
 

Mert

Member
I remember I seen a topic in old forums about this issue, and I also remember that he solved this issue.
(Just saying, try to search old forums : gmc.yoyogames.com)
 

FrostyCat

Member
You have to place a mutex on the client-bound socket so that only one of the many threads write to it at the same time. In Java, you can do it with synchronized or java.util.concurrent.Semaphore.

You will still get a random choice between 1S-1F-2S-2F or 2S-2F-1S-1F, because mutexes can't predetermine execution order. Fortunately, you won't have to deal with any of the other bum combinations, because mutexes can prevent unwanted race conditions.
 

CodeManu

Member
Yes. You need to make sure this does`t happen.
You have to place a mutex on the client-bound socket so that only one of the many threads write to it at the same time. In Java, you can do it with synchronized or java.util.concurrent.Semaphore.

You will still get a random choice between 1S-1F-2S-2F or 2S-2F-1S-1F, because mutexes can't predetermine execution order. Fortunately, you won't have to deal with any of the other bum combinations, because mutexes can prevent unwanted race conditions.
Yes! That was it! Threads were being a mess and that was making the data arrive mixed up. I've put a lock and it works beautifully! Thanks guys you are the best <3
 
Top