GMS 2 Snapshot Interpolation

Discussion in 'Advanced Programming Discussion' started by MudbuG, Sep 5, 2018.

  1. MudbuG

    MudbuG Member

    Joined:
    Jun 21, 2016
    Posts:
    42
    I would like feedback on the following code that I am using in the step event of a network object that implements (or attempts to implement) snapshot interpolation.

    While the results are much better than just raw position updates, there is still some jitter. I am not sure if the jitter can be completely eliminated, or if I have to live with it.

    Some things to note:
    • time_offset was arrived at by a clock sync message to the given peer. I sent current_time to the peer, received the peers current_time, calculated round trip and offset from reply.
    • the object being interpolated is a remote tank. Tank movement is time-based vector movement with smooth acceleration and deceleration (which probably complicates the jitter issue)
    • I get about a 200 MS RTT with GameSparks (100ms lag), and I am targeting between 120ms to 150ms delay for the interpolation.
    This is my first attempt at interpolation. I am wondering if I am missing something a bit (probably), or if a different approach would be better for this situation.

    Code:
    /// @description Interpolate Movement
    
    // must have at least 2 states in history in order to interpolate
    if (ds_list_size(interp_buffer) < 2) return;
    
    // determine target render time
    var renderTime = current_time - (interp_back + time_offset); //time we want to show the position for (adjusted to remote time clock)
    
    // find most recent state whose time that is <= our target renderTime
    var index = ds_list_size(interp_buffer) - 1; //start at the end, it has most recent state, and iterate backward
    while (true)
    {
        var item = interp_buffer[|index];
        var itemTime = item[?"time"];
        if (itemTime <= renderTime)
        {
            break;
        }
        
        index--;
        if (index < 0)
        {
            //reached end of list, can't interpolate, exit the script
            exit;
        }
    }
    
    // make sure we have 2 states
    if (index >= ds_list_size(interp_buffer) - 1)
    {
        // can't interpolate... not enough states. 
        // optional: could just set to this position, or try to extrapolate
        exit;
    }
    
    //
    // Interpolate between the 2 positions we found
    //
    var p1 = interp_buffer[|index-1]; //target renderTime or older
    var p2 = interp_buffer[|index];   //newer than target renderTime
        
    var p1Time = p1[?"time"];
    var p2Time = p2[?"time"];
    var x1 = p1[?"x"]; var x2 = p2[?"x"];
    var y1 = p1[?"y"]; var y2 = p2[?"y"];
    var velocity_x1 = p1[?"velocity_x"];
    var velocity_x2 = p2[?"velocity_x"];
    var velocity_y1 = p1[?"velocity_y"];
    var velocity_y2 = p2[?"velocity_y"];
    var totalTime = p2Time - p1Time;
    var pos = (renderTime - p1Time)/totalTime; //find position between 0 and 1 so we can use lerp
        
    // set actual lerped position (x and y) and angles
    if (abs(x1-x2) > 1) x = lerp(x1, x2, pos);
    if (abs(y1-y2) > 1) y = lerp(y1, y2, pos);
    
    //velocity is only used to set image_angle for now.
    velocity_x = lerp(velocity_x1, velocity_x2, pos);
    velocity_y = lerp(velocity_y1, velocity_y2, pos);
    image_angle = point_direction(0, 0, velocity_x, velocity_y) - 90; //-90 used becuase sprite is not oriented properly
    
    // directly set turret angle without interpolation
    turret.image_angle = p2[?"turret_angle"];
    
     
  2. Simon Gust

    Simon Gust Member

    Joined:
    Nov 15, 2016
    Posts:
    3,044
    hmm, quite hard to find issues.
    A question though, you use
    Code:
    var pos = (renderTime - p1Time) / totalTime;
    
    and
    Code:
    var renderTime = current_time - (interp_back + time_offset); //time we want to show the position for (adjusted to remote time clock)
    
    Is interp_back a state of time or is it a delay?
    At the end of the day, renderTime should be a state of time, just like p1Time is the last state of renderTime.
    If any of those is an amount of time, it will not work correctly because current_time will never reset and is an absolute state of time.
     
  3. MudbuG

    MudbuG Member

    Joined:
    Jun 21, 2016
    Posts:
    42
    Thanks for taking look!
    interp_back is the delay in milliseconds. renderTime is a state of time.
     
  4. Simon Gust

    Simon Gust Member

    Joined:
    Nov 15, 2016
    Posts:
    3,044
  5. Simon Gust

    Simon Gust Member

    Joined:
    Nov 15, 2016
    Posts:
    3,044
    I made a little simulation about this and also learned a lot.
    What I did is have the server send a packet every 12 frames, the client would then have a delay of 4 to 5 frames.
    But I kept the interpolation simple
    Code:
    if packet has arrived and there are at least 3 packets in queue
    set timer to 0
    
    if timer < server send rate
    timer += 1
    
    x = lerp(x1, x2, timer / server send rate)
    y = lerp(y1, y2, timer / server send rate)
    
    The results were pretty smooth.

    Now, the problem comes when the player's ping starts being inconsistent:.
    If the first packet had a ping of 3 and the second a ping of 20, the player would jitter and teleport around.

    In addition to that, if the client ping is greater than the server send rate, the client falls behind on reading packets causing permanent delay of the client. To prevent this, I suggest the client telling the server to stop sending packets if his ping is too high.

    So I tried to add the average ping to the server send rate, but that just made it worse.
    And what I think of that is that you shouldn't depend on the client's ping.

    I don't know yet what to do against inconsistent ping but I hope my experience helps you.
     
    Last edited: Sep 6, 2018
  6. MudbuG

    MudbuG Member

    Joined:
    Jun 21, 2016
    Posts:
    42
    Interesting.

    What would you think about limiting the change in velocity each frame? I am hoping it would might smooth things enough to account for changes in ping rates and create the illusion of accuracy.

    Just to confirm what I think I see... are you using steps for the time unit for time variable?

    I had a "aha" moment. I started checking the sequence of packets that I receive against the last packet received. Sometimes the most recent packet in the buffer is newer than the one I am receiving. In that case, I just toss out the received packet instead of putting it in the buffer. I am now able to get fairly smooth results sending only 10 packets per second, and interpolating back 150ms.
    Additionally: To eliminate clock sync from the mix, I started interpolating against time received. Both of those things together make a big difference.

    Summary of 3 key improvements that resulted in smooth interpolation:
    1. Only accept snapshot packets in the proper order, ignore out-of-order packets (I read this and should have known it)
    2. Use received time to interpolate against instead of attempting to adjust for remote time clock (that doesn't mean to ignore latency, I am still keeping track of it, but not using it here)
    3. Set the interpolation delay far enough back to make up for variation in lag ("ping time")

    Further Improvements that I think can be made, but are beyond the scope of my project --- maybe someday
    • Dynamically adjust interpolation delay based on average packet latency
    • Disconnect the peer if latency becomes too high
    • Extrapolate when no "future" packets are available (hopefully that should be rare if interpolation delay is set close to correct)

    I am sure this can still be cleaned up more, but is the version that is working quite smoothly for me now. The following code is in the step event of the object being interpolated:
    Code:
    /// @description Interpolate Movement
    var bufferSize = ds_list_size(interp_buffer);
    
    if (bufferSize < 1)
    {
        return;
    }
    
    if (bufferSize == 1)
    {
        //just get the latest snapshot, and set position
        var p = interp_buffer[|ds_list_size(interp_buffer)-1];
        x = p[?"x"];
        y = p[?"y"];
        image_angle = p[?"angle"];
        turret.image_angle = p[?"turretAngle"];
        return;
    }
    
    // determine target render time
    var renderTime = current_time - interp_back; //time we want to show the position for (adjusted to remote time clock)
    
    // find most recent state whose time that is <= our target renderTime
    var index = ds_list_size(interp_buffer) - 1; //start at the end, it has most recent state, and iterate backward
    while (true)
    {
        var item = interp_buffer[|index];
        var receivedTime = item[?"receivedTime"];
        if (receivedTime <= renderTime)
        {
            var msg = "Interpolation snapshot found at index: " + string(index) +
                  " receivedTime: " + string(receivedTime) + "   renderTime: " + string(renderTime) + "  diff: " + string(renderTime - receivedTime);
            show_debug_message(msg);
            break;
        }
      
        index--;
        if (index < 1)
        {
            //reached end of list, can't interpolate, exit the script
            show_debug_message("Interpolation time not found");
            exit;
        }
    }
    
    // make sure we have 2 states
    if (index >= ds_list_size(interp_buffer) - 1)
    {
        // can't interpolate... not enough states.
        // optional: could just set to this position, or try to extrapolate
        exit;
    }
    
    //
    // Interpolate between the 2 positions we found
    //
    var p1 = interp_buffer[|index-1]; //target renderTime or older
    var p2 = interp_buffer[|index];   //newer than target renderTime
      
    var p1Time = p1[?"receivedTime"];
    var p2Time = p2[?"receivedTime"];
    var x1 = p1[?"x"]; var x2 = p2[?"x"];
    var y1 = p1[?"y"]; var y2 = p2[?"y"];
    var totalTime = p2Time - p1Time;
    var pos = (renderTime - p1Time)/totalTime; //find position between 0 and 1 so we can use lerp
      
    // set actual lerped position (x and y) and angles
    if (abs(x1-x2) > 1) x = lerp(x1, x2, pos);
    if (abs(y1-y2) > 1) y = lerp(y1, y2, pos);
    
    //velocity is only used to set image_angle for now.
    image_angle = p2[?"angle"];
    turret.image_angle = p2[?"turretAngle"];
    
    Here is the code that handles receiving the packet and ignoring out-of-order packets. I am using sender time for that, which acts like a message sequence. Message sequence could be used instead.

    Code:
    var remoteTank = find_or_create_remote_tank(peerId);
    
    // make sure this packet is actually newer than the most recently received one
    var bufferSize = ds_list_size(remoteTank.interp_buffer);
    if (bufferSize > 0)
    {
        var lastSnapshot = remoteTank.interp_buffer[|bufferSize-1];
        if (lastSnapshot[?"sentTime"] >= packet[?"sentTime"])
            return;
    }
    
    // add new snapshot
    var snapshot = ds_map_create();
    snapshot[?"x"] = packet[?"x"];
    snapshot[?"y"] = packet[?"y"];
    snapshot[?"angle"] = packet[?"angle"];
    snapshot[?"turretAngle"] = packet[?"turretAngle"];
    snapshot[?"sentTime"] = packet[?"sentTime"];
    snapshot[?"receivedTime"] = current_time;
    ds_list_add(remoteTank.interp_buffer, snapshot);
    if (ds_list_size(remoteTank.interp_buffer) > remoteTank.interp_buffer_max_len)
    {
        var sn = remoteTank.interp_buffer[|0];
        ds_map_destroy(sn);
        ds_list_delete(remoteTank.interp_buffer, 0);
    }
    
     
  7. Simon Gust

    Simon Gust Member

    Joined:
    Nov 15, 2016
    Posts:
    3,044
    Yes, I do that a lot.
    It works for the basics but it you try to read latency, not so much.

    Also, here's how am going to do position syncing if you are interested:
    I stream the inputs of my player for 10 frames, send them over the network and the client then simulates it with the inputs received.
    This way, my player objects and dummy client objects are actually just the same object, except one reacts to my inputs and the other reacts to the inputs from the packets received.
    To save memory on the input-streams I bitmask every input into 1 value where each bit is a different key press, so I only have to send 10 values for 10 frames of streaming.

    I have not tried this in practice, but it saves me from needing to interpolate at all.
     
  8. MudbuG

    MudbuG Member

    Joined:
    Jun 21, 2016
    Posts:
    42
    Sending inputs and simulating at the destination is another thing that I intend to try at some point.
    Do you send position updates occasionally in order to keep the remote/ dummy insurance from getting too far out of sync?

    Also to keep track of frame time, do you just use a counter and increment every step, or is there some other way to get frame number?
     
  9. Simon Gust

    Simon Gust Member

    Joined:
    Nov 15, 2016
    Posts:
    3,044
    I probably have to, because if one packet drops, the dummy player is permanently desynced not only time wise but also position wise.

    For the frame counter I use my own timer
    Code:
    global.tick++;
    if (global.tick mod 12 == 0)
    {
        // send positional update
    }
    
    This will send a packet every 12 frames without having to reset the timer due to the power of modulo.
     

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice