How to handle 5000 or more players on a JAVA server? Open discussion.

Discussion in 'Advanced Programming Discussion' started by JesterOC, Jul 21, 2017.

  1. JesterOC

    JesterOC Member

    Joined:
    Jun 27, 2016
    Posts:
    16
    Open Discussion
    Here we discuss methods for managing large online player count on a Java server.

    What is a good method?

    Currently I have no method implemented, however i would like to use NIO Selector in java but from what ive read its complicated and couldn't find a working example.

    My game's Java server is made from a JAVA server example I found online, ive implemented google play games for sign in and remote push notifications are working, however while trying to fix an issue* i read that a single thread per client is likely to lag on a small server and that using NIO selector is better. This leads me to wonder about how other game makers are handling such things...

    I'll share my Java Server Code later, i'd share my Game project as well but it uses some android extensions i got on the marketplace, and idk what the deal is with redistribution of that stuff, but pree sure its illegal.

    The Java
    might help someone though. :)

    EDIT: I'M STILL TRYING DIFFERENT METHODS...
    I don't think anyone wants to look at the mess I've made though. Currently my Strings aren't read out from my GM client, everything else is working as expected including writing string to server.

    *I'm Done... I registered and logged into my server with username and googleID, I sent some push messages to my phone... after selecting myself from a list of players that were in the current room. I send some to my mum's phone aswell. most of the time it worked... sometimes i got duplicate messages on the client... (no sound either and i send another Alert Dialog Message from modified textinput Extension saying sent gift which is a no show. (could be because of the incoming one? Edit: I can confirm, INCOMING PUSH MESSAGE DIALOG OVERRIDES?/CLASHES WITH MY OTHER EXTENSION DIALOG(s?) ! FFS, Anyone know what's going on here? is it a bug? .)
     
    Last edited: Aug 1, 2017
  2. orange451

    orange451 Member

    Joined:
    Jun 22, 2016
    Posts:
    108
    I've been using Java as my main programming language for the last 6 years. There shouldn't be any reason a server couldn't handle 5000+ socket connections.

    A good practice is to define a max amount of players for your server, say 5000 or so (make this number not changeable until you restart). At launch, create a pool of 5000 threads that you will use for networking. It is very useful to thread out your networking in the long run, so one bad apple wont starve everyone else out. Though it's also important to create a pool. Java can't garbage collect system threads, and you'll eventually run out of memory if you create a new thread per player.

    In my current game I do this. It doesn't take any fancy sort of setup for the actual networking. You could use the old-style of sockets (java.io) or the new version (java.nio). Both work quite well, and are left entirely up to preference (streams vs buffers).

    If you go with java.io, look into the ServerSocket, Socket, BufferedInputStream, and BufferedOutputStream classes.
     
    andev and JesterOC like this.
  3. JesterOC

    JesterOC Member

    Joined:
    Jun 27, 2016
    Posts:
    16
    Thanks for Joining the discussion.

    The example i was using used IO, and it had one network object per player, and Player class extends Thread

    I will have to look into thread pooling.. (I'm fairly new to Java) - My first attempt to make a server was going well until I wanted to upgrade the IO to NIO,
    It's been a few days going and I've only got the server accepting connection and reading strings and numbers sent from the game maker client... But I must've mucked up somewhere cus when I go to write the buffer like this:
    (Similar to what it was before except i've moved net and write buffer to the server class for time being)

    Code:
    net.clearbuffer(Write);
    net.writebyte(5,Write);//Send init success
    net.SendSock(p.sock,Write);
    It gets all the way to my switch for packet header... except its always ZERO. :(

    Ive tried changing some things to return the 5 i send, but it hasn't worked so far, get BufferOverflowException and Index out of bounds whenever i change anything. :/

    I'll be posting my working IO and Partially working NIO java servers when i have some time to fix up the code (it's a bit messy cus of me, and it's probably not the greatest as I'm still learning and haven't added the pooling of threads yet.)

    @orange451
    Can I PM you for a chat about Gamemaker and Java sometime?
    It'll probs only be if I get really stuck... cus I don't like bothering anyone.
     
  4. orange451

    orange451 Member

    Joined:
    Jun 22, 2016
    Posts:
    108
    I have skype if you want to chat: orange4513
     
    JesterOC likes this.
  5. FrostyCat

    FrostyCat Member

    Joined:
    Jun 26, 2016
    Posts:
    3,405
    With 5000 simultaneous connections planned for, I'm pretty sure you're at least approaching the point where infrastructure becomes the main bottleneck. You should look into load-balancing the incoming connections across several smaller sub-servers, regardless of the language you plan to write the server in.
     
  6. JesterOC

    JesterOC Member

    Joined:
    Jun 27, 2016
    Posts:
    16
    I was Hoping to get away with just one server, keeping most of the variable syncing to a minimum... getting rid of player x,y variables, and just telling players what "SPACE" the player is in. what variables a player has when you request to see their player profile/room/whatever... the main multiplayer aspect was gonna be gifting... just a little rpg game with gifts via push notifications and visiting other players profile/room/whatever... that's it really.

    Should be possible, no? (It sucks i have no way to test 5000 simultaneous connections)
     
  7. MishMash

    MishMash Member

    Joined:
    Jun 20, 2016
    Posts:
    366
    Would definitely echo what FrostyCat is saying, many routers wouldn't even necessarily be able to handle that level of simultaneous connections. More importantly, it is a horrendously large load for a single server. If you are doing anything remotely complex, then its going to get expensive very quickly.

    Even at an operating system level, listening to that number of sockets and waiting/checking for data is going to be slow, unless you have an incredibly powerful networking card and a similarly powerful computer.

    A simple distributed server setup would have a number of connection nodes that players can connect to, lets say in this case, you would want around 10 connection nodes for 5000 players, meaning each node would handle 500.
    - Each of these nodes would connect to your primary server, however no players connect directly to the primary server.

    - Local nodes simply bunch together traffic, and forward it in larger packets to the primary server. (if your game is simple as you said, you can get away with having your primary server do all the processing, but you would want other servers to offload the bottlenecks introduced from the connections.) The good news is packet size isn't really an issue these days, as most network connections have good bandwidth, the limit is simply volume and saturation.

    - Assuming you are reasonably experienced at networking and have designed a good protocol, then forwarding this data shouldn't be an issue. You can extend your protocol so that each packet also gets a encoded with a little bit of information about the player it is from (as the main server wont know this if it cant cross-check against the specific socket).

    - You will also need to forward the connections list and keep the primary server informed.

    --------
    If your game is more complicated, then you need to also consider distributing the server work load. This gets more complicated. What this would involve is having each node perform all the work it can locally (lets say if players on the same node are interacting with each other, then the local node can perform all of these interactions on itself, then notify the main server with any changes later on), if however, two players are on different nodes, you would then need to branch off to other nodes. This can either be done with inter-node connections, or you can use the main server as a bridge between the two.

    If you are going to be creating a server, i'd strongly recommend Node.JS over Java (if you are gunning for something cross-platform), as you don't have to worry about threading, or NIO. Javascript has this beautiful thing called the event loop. What it means is that similar to GM, events run one after the other. So for example, any time you receive data from a client, this will internally fire off a javascript event, but it wont steal all of the processing, it will only run when it needs to run. Similarly, you can use timeouts or promises so that events only run when they need to run, rather than having to "poll" a connection to see if it has submitted any data. The GameMaker networking infrastructure actually works in the same way, which is beneficial for supporting large numbers of connections without starving the computer of resources.
    The downside being that you cant utilise all your computers cores unless you delve into threaded programming.


    A few general tips too:
    - Java networking can sometimes fragment packets (Had a pain with this with my University group project where half a packet would arrive in the buffer, then the second half of it would come a little bit after), therefore, it is really good practise to write the size of a packet to the start of the packet, so that if for whatever reason, data is missing, you can detect that, rather than over-running your buffer. (You can then wait until more data comes in before continuing to read that packet).
    - on that same note, if you encode the size in each packet, then if for whatever reason, you have a mistake in your net code that reads the wrong amount of data, at the bottom of your packet loop, you can "correct" the reading position by setting it to the position at the start of the last packet, plus the size. < this makes networking issues far easier to debug, and also reduces the number of random crashes you might have.
     
    dj_midknight likes this.
  8. JesterOC

    JesterOC Member

    Joined:
    Jun 27, 2016
    Posts:
    16
    Node.JS sounds like a dream.. but ive only just realised my dream of a working java server... so i dont wanna recode network stuff in a language i dont know :/

    it took me 42 server versions and about 79 client versions to get login to server with google login and push messages from server working... so im kinda exhausted nd depressed now. :(

    i fixed the duplicate push messages by removing the sent "gift dialog" that was sent immediately afterwards from sending push msg. but idk what happens if one fires when im in another dialog. maybe it will overwrite it nd ill have duplicate gifts received.

    And as much as id love to use multiple server nodes. i just cant afford that many vps... i barely afford my current one.

    So i guess my quest ends here...
    ..or maybe i could make a client-run server and link to it from play store? people could set up their own multiplayer worlds, handle servers settings and mine can just maintain the ip addresses of the servers people set up? Maybe im just not thinking correctly atm...

    I'm not really that great with JAVA or GML when it comes to networking... Im okay as long as its "send this as type" --> "receive this as type" , but when it gets to having to handle connections... pack data into buffers and all the technical stuff i start getting lost... thats why it took me so many versions i think... that and i restarted halfway thru.

    Anyone know of a completed online gamemaker game? id liek to see that its possible before I attempt to continue.
     
  9. madfast

    madfast Member

    Joined:
    Nov 28, 2016
    Posts:
    14
    There are a million what-if's, it-depends and other bits, but I hope this gives you a working idea for a solution in your budget.

    1) User connect/authenticate via HTTP/HTTPS - Give user a token - assign token to session - It's okay if this is slow. - I'd make this some website somewhere, do not put it on your game processing box. It can back-channel the authenticated token information and IP to the game system.
    2) Game sends all updates from an authenticated client, to your server IP, via UDP to port - Port knows "valid" IPs from those that authenticated - only allow authenticated IPs
    3) Your UDP data frame will be super small - you have to allow for all of the issues mentioned above - However, set Fragmentation at IP level to not allowed at the perimeter firewall - thus making sure you never get packets that are fragmented at IP layer. This does not handle UDP fragmentation. So you have to "KNOW" and plan ahead your maximum size of a UDP frame that will not be fragmented by the MTU on the network stack. I recommend never sending more than ~384 bytes, I'm already subtracting a lot of overhead. You want it fast, and you want it small. You can do research, but if you send specificly larger frames, then routers on the internet may fragment those packets.

    4) The data - bit packing - learn how to mask and shift bits to build these payloads.

    unsigned 8 bit number - size
    Some array of bytes for your token
    Int16 command - this needs to be a bit mask, utilize each bit in the byte, and allow complex states (you should have a priority flag here for priority updates)
    Additional data for your update

    Flow:

    1) Receive packet from the UDP socket listener
    2) This is the most IMPORTANT step - Send the damn packet to another thread which is blocked and waiting for the packet - and other data as necessary - Verify anything you allocate here, is freed later - At this point it should only be a few hundred bytes tops so no big deal - you can allocate that here for the moment, later you want a preallocated heap that you can micro manage so you have no performance overhead on alloc/free
    3) Start listening for the next packet on that port

    Network Dispatcher:
    1) Packet from listener thread - New packet arrived yay
    2) Is this IP in your session list? If no, drop packet
    3) Check size of payload, it has to be equal to or smaller than the actual frame, don't trust this value if it's larger than the number of bytes you have in hand as that is how overflows happen
    3) Extract token
    4) Is this token (session from a valid user) valid, if no drop packet
    5) If this token belonging to the person at this IP? If no, drop packet
    6) If you get here, it's good "enough" - can work on making it more secure later if you get to this point
    7) Check for priority flag - if set, send to priority thread, otherwise send to non-priority thread

    Priority Thread

    1) On receive, check some payload data for specific flags/values that say this is a movement, or a shoot at, or whatever your update is that has to be fast

    Non-Priority Thread

    1) On receive, check some payload data for specific flags/values ... this would be more like a msg sending to someone or a channel, versus a movement update

    This will all be relatively fast, the problem you will encounter is when you want to logically send updates to a few, so your packets going out will need some similar logic and consideration.

    This is all before you are even doing any server side logic. I'm not sure the reasoning behind 5000 active connections, but if you consider 5000 current users, then you are looking at upwards of at least 2 updates a second, preferably more. At even this small amount of traffic it's 240K bytes per update (assuming all 5000 update) - so if you want 10 per second, you can try, that's about 2.4 mb per second coming in, which is easilly processed from a networking perspective in this way.

    With all respect to the 5000 threads approach, this will create a concurrency nightmare. Unless you have a requirement for pipeline separation of data (security) there is no reason to turn data into code for purposes of processing. A computer does not run faster in that model and any message pump properly implemented as described above will have better performance by literally 10x, 20x, 30x, 40, etc. The number of task switching requirements between threads, even with Xeon processors of the highest cos, cache and speed, will still be slow in these models, and it will eat up your cores, literally 30+% will be spent task switching between threads. You can optimize concurrency when you have a pipeline design working on a system, to then add 1, 2 or so more threads to allow distributed processing/concurrency/parallelism of the messages, and perform dispatching on a dedicated core (be it a virtual or real core). Truly work out Amdahl when you are doing the model. https://en.wikipedia.org/wiki/Amdahl's_law

    You need to brush up on the threads, semaphores, mutex and queues. You use threads to separate processing so you don't block tasks when possible, you use queues and mutexs so when you pass data between threads you don't have synchronization issues. You use the semaphores and mutex to stop reentrancy where applicable. In early 2000s, I was doing something similar, with ARP packets though (don't ask), but I was sitting around 8 million frames per second with custom content. Even with Java this performance is doable, you just have to do the thread staging and dispatching properly.

    You only need to process 2.4mb of data per second. Even if you consider that you have 2 cores (or MORE!) then you need to dispatch 1.2mb of data from the network per second to the processors. If you have 4 cores, then that's 600K per second. Assume most of those are critical "movement" updates from which you will also be streaming "outbound" traffic back to the user. As recently as a year ago I had about 6 giga texels of throughput per second on a GPU, without any "trickiness". From just a CPU power perspective, you should be able to run this server on a laptop in the park and manage 50,000 UDP packets per second.

    1) Start with just one client and server - get it working - model async thread pumps as I described
    2) Make a client that spams UDP packets to your server with some update or whatever, and measure/watch the performance.
    3) You're limit will be when you start doing processing on the packets - which is another discussion, but using this model you should be able to infer

    Absolutely achievable.
     

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice