Networking bandwidth – Gonna need a bigger pipe

Not much has changed recently that is directly visible in game, instead I have been doing a lot of work under the hood to improve networked multi-player.  Primarily to reduce bandwidth, but also to make things a bit smoother on the client side during packet loss situations and so on.  There is still much more to do however.  For example, I don’t yet have client-side prediction for the locally controlled player working in a way that I’m happy with.  Also, I need to implement network culling so that clients only receive data for objects that are near them in the world.  Anything I can do to reduce bandwidth is a must.  I may end up having to bypass Unity’s networking altogether, as it just doesn’t give you direct control down to the byte level over what gets sent, and there seems to be a large overhead per networkview state sync, and per RPC call.

This whole effort was to help me evaluate if networked multi-player is even viable for a game like mine with a large number of dynamic objects, and whether my current server-authoritative model is the best fit.  This has a huge impact on the game design as a whole, so is important to figure out.

As it stands right now, each client sends roughly 1000 bytes / sec of controller input to the server, and receives about 1000 bytes / sec per player connected (including themselves of course) for position, animation state and so on.  Add to this up to 1000 bytes / sec received per currently moving rigidbody in the game (i.e. a single solid construction – a construction made of two things jointed together counts as two rigidbodies).  You can see it soon starts to add up.

Things are even worse for the server – it has incoming input data from every client, and has to send out object states to each client.  Assuming 16 players with 100 moving rigidbodies, that’s 16 * (16 +100) * 1000 = ~1.8Mb / sec outgoing bandwidth for the server!  The actual data I’m sending per rigidbody should only add up to 780 bytes / sec, so the per networkview overhead clearly isn’t helping here, but even assuming this can be solved, it would still come out to ~1.4Mb / sec in the 16 player example.

Some of my options are:-

1. Further bandwidth improvements.  For example, by sending position / orientation state for rigidbodies that are part of the same construction relative to some common root, I think some redundant data can be eliminated.
2. Reduce the maximum number of players.
3. Cap the number of simultaneously moving rigidbodies.
4. Change to an authority switching networking model (where each client simulates physics for objects local to themselves, and the server is there to assign authority and resolve authority conflicts when players are near each other).

Option 1 will help the general case, and definitely should be done, but it won’t help the worst case where every rigidbody happens to be a separate construction.  The second two options aren’t particularly appealing, but may be a necessity.  Option 4 would be a lot of work to implement, and has a big impact on game design because it is completely open to cheating and only really makes sense for co-operative play.

I need to do some tests with similar games such as Garry’s Mod (based on the source engine, which I know is using basically the same networking model as I am right now), to see how I stack up in comparison, bandwidth-wise.  Regardless I think for now I’m going to stick with the current server-authoritative model, and assume a small number of players.  Maybe in the future if this game takes off and the effort of implementing an authority switching model for co-operative play makes sense, I’ll do that, but that would be a way off I think.