[GH 120] Desync for users in sessions with high player count and latency

Originally created by: @Stellanora64

Describe the bug?

For primarily Australian, Japanese, and New Zealand users that are on a US headless servers or user hosted session with a higher player count (around 10 to 15 or more) Desync begins to occur.

This is not restricted to the users listed above but usually occurs when latency or ping is above 100 - 150ms (note this as reported by Resonite through protoflux, but is currently half the value it should be after some testing by sending packets directly to a host user through other means. So more accurately would be 200 - 300ms, an issue for this has been reported here: #34)

Note:

  • It appears to be able to recover if there is not much active development in the session (inspector usage, flux usage etc) and the user count is less than 15 users.
  • Once this threshold is reached though, and even some develop to the world is being done (or none, if the user count is above 20) network packets will queue at a rate of ~200 packets per seconds, forever, until the session has gone below the conditions listed above.
  • There is almost always a spike of queued packets when a new user joins (roughly 300 - 1000 queued packets), but this recovers quickly when the session is under the conditions listed above.

Desync begins to affect primarily:

  • Inspector panels taking 100 to 200 seconds to show any slots. (This being due to inspectors being generated by the host and thus desynced users need to wait for the host to generate the UI)

  • This time is directly proportional to the amount of queued packets a given user is experiencing. ( Roughly every 1000 QP is the equivalent of 10 seconds of delay. However this is just an estimate as this ratio varies on ping and other factors like bandwidth usage)

  • Objects other users have grabbed and moved only catching up after around 100 seconds (the same time it takes for inspectors to catch up, again dependent on queued packets).

  • other users voice modes. for example if someone, that is not the desynced user, changes their voice mode to mute they will not be muted for the desynced user until the desync has catches up which they will be muted mid sentence. This being due to mute not being controlled through the audio stream, but instead just disables the audio output component when a user is mute. This can lead to unintentional instances when a user that is desynced can hear a private conversation outside Resonite, or when two users are in whisper.

(note that their is no desync between player to player communication, like talking to another user is not affected by desync as well as their avatar's movements)

  • objects other users have spawned only appearing for the desynced user once desync catches up.

  • any changes other users have made, like changing a material, mesh etc.

  • When a user changes their avatar, they appear frozen until the desync catches up. This also stops all player to player interaction that isn't normally been affected by desync like talking and avatar movement/player position, but only for the user that has changed their avatar. They essentially become frozen until the desynced user's computer receives the packet that the other user changed avatar.

To Reproduce

Reproducing this issue may be hard to replicate for users other than Australian and New Zealand users but is a consistent issue in creator jams where the player count is at around 15 or more and the headless is US, and there is constant active development happening.

the main things needed to reproduce the bug are to be connected to a session that is generally a very far distance from where you are playing (a VPN may help with this) and to have the session have 10 or more users, and have active development happening (or have the user count that is equal to or above 20 users, as at that point development or not doesn't really matter).

Expected behavior

To have no desync in sessions that are in a different region to users that join, and for there to be no desync with a high player count session.

Screenshots

No response

Resonite Version Number

2023.10.13.743

What Platforms does this occur on?

Windows, Linux

What headset if any do you use?

Desktop, HP reverb G2 (although, headset doesn't affect this issue)

Log Files

I will provide a clean log this Sunday during the creator jam.

Additional Context

No response

Reporters

Resonite: Stellanora

Discord: stellanora