Author Topic: Server Lag : what it is and isn't....  (Read 15156 times)

Offline Lambchops

  • Ogre Mage
  • ********
  • Posts: 1541
    • View Profile
Re: Server Lag : what it is and isn't....
« Reply #30 on: October 19, 2016, 02:18:02 PM »
Appreciate if someone can offer some libraries for keeping UDP interchange between clients or w/e to integrate into C++ project.

I think just standard 'socket' 'connect' and 'sendto' etc. should do it for anything I've been talking about... they're about the only API calls that are identical on Windows and UNIX...
its gooder to hax hard and NEVER get caught!

Offline iL

  • Administrator
  • Ogre Mage
  • *****
  • Posts: 1650
    • View Profile
Re: Server Lag : what it is and isn't....
« Reply #31 on: October 19, 2016, 06:32:19 PM »
One way to fix this *COULD* be to change the route by using a fast proxy or VPN. A VPN establishes a connection over HTTPS between the client and the VPN provider, from where the data would most likely be routed through a different set of servers to the other players.
Interesting if that's posible to make something like NAT substitution for such problem clients. I mean some kind of wrapper between war2 client and network adatper. Say, the problem is between 1.1.1.1 and 2.2.2.2. War2 from 1.1.1.1 sends UDP to 2.2.2.2, that wrapper translates 2.2.2.2 to 3.3.3.3 (another client which connections to 1.1.1.1 and 2.2.2.2 are faster) like kind of vpn. 3.3.3.3 translates traffic back to 1.1.1.1 after receiving answer from 2.2.2.2.
Such kind of distributed UDP antilag-network. With dynamic scanning network performance between each pair of clients and finding the best routes.
Such solutions should be implemented somewhere already. Would be great to get such opensource solution and use for our tasks.

Yes this would be great, but it would not be a small project. Say you built in a layer (storm.dll somewhere, I guess?) that detected packets with an extra custom header designating that they should be redirected elsewhere then had it just strip that header and re-transmit the packet. Then you would have to have each client test its connection to each other client via a third and decide on the best solution. For an 8 player game this means each of the 8 players doing 36 individual load tests. You would have to decide on thresholds for proxying and repeat the test because if everyone decided to use a single client with a nice connection to proxy everything, that client would suddenly have 49x the nework load and its performance would drop..... it's a "movable feast" as my father would say... but it would be very nice once implimented.


Oh, and the gold standard would be this happening dynamically, in game... like when a player has an intermittant problem connecting to some players, it would detect this in real time and start proxying through another client.....  Hey, you code it ... I'll cheer you on ;)

--------------------------

In effect it's a kind of VPN ... without the P.

 But i think (not sure)  VPN's all work through a server, not peer-to-peer, don't they? I'm not aware of any open source project that does this, but would love to hear about it if there is.

---------------
(edit) ..... (again lol)

no, by the looks of it you can configure them to do all sorts of stuff...
check this out maybe.
... very late here, I need to get some sleep.
Of course it's not a small project, but i hope to link it from several separate parts. Not unresolvable task for experienced network programmer (in theory, if we find one of them).
Part 1: I'm not about to make some storm.dll research and inject code there. I'm talking about some solution to put in the middle between any application (war2 in our example) and network adatper. I don't know such solutions, but they should be released already. The easiest implementation i see is hooks for several winapi network functions.
So, we can just make some additional traffic between clients or maybe from all clients to a server. Maybe just additional tcp or udp connection to different port. Now, we know all latencies between all clients talked to each other.

Part 2: Another part is additonal headers to UDP packets for rerouting. Quite easy part.

Part 3: Connection management. Receives all the latencies continuously, building and rebuilding it's own map, returns the map for optimal network every moment. Receives all the clients and latencies from part 1 and enforces management using part 2.
Of course very difficult to write this from scratch, but i trust there are some ready solutions to resolve the same problem.

I'd look into distributed networks, p2p networks, VoIP technologies, etc.
VoIP is very active direction for current IT-networking, it uses UDP for it's traffic and it's main task is to make as good connecting as possible. Should have some ready to use solutions.
Need help to translate War2Combat to German, French, Italian, Polish or another language: http://forum.war2.ru/index.php/topic,4728.0.html
Please, contact me if you are interested in that.

Offline iL

  • Administrator
  • Ogre Mage
  • *****
  • Posts: 1650
    • View Profile
Re: Server Lag : what it is and isn't....
« Reply #32 on: October 19, 2016, 06:50:27 PM »
i got impatient and looked into it myself... i think i was kinda on the right track:

Skype made a technique popular that is called firewall hole punching....

Nice one. Thanks for that, I didn't know that term.
Yes, hole punching is the thing known as lat trick in war2. Some other related terms are: STUN or even ICE: combination of methods to maximize the chance for successful connection between all the clients.
Would be great to find ICE implementation for our purposes.
As i see, a C++ class/library to include into war2 loader.

Just thouhgt: would be great to find good network programmer/engineer for war2 project:
We could handle the whole war2 network behavior ourselves: to guarantee all the UDP connections between all the cilents, to manage and find optimal routes for that connections, or even to reroute if clients will suddenly change their IP during the game.

UDP over internet is far from perfect and has lots problems. So, some kind of layer between raw war2 UDP and internet (like distributed UDP network between war2 clients) based on new technologies and implementation is what we need.
« Last Edit: October 19, 2016, 06:57:22 PM by iL »
Need help to translate War2Combat to German, French, Italian, Polish or another language: http://forum.war2.ru/index.php/topic,4728.0.html
Please, contact me if you are interested in that.

Offline Lambchops

  • Ogre Mage
  • ********
  • Posts: 1541
    • View Profile
Re: Server Lag : what it is and isn't....
« Reply #33 on: October 20, 2016, 04:55:02 AM »
As i see, a C++ class/library to include into war2 loader.

Just thouhgt: would be great to find good network programmer/engineer for war2 project:
We could handle the whole war2 network behavior ourselves: to guarantee all the UDP connections between all the cilents, to manage and find optimal routes for that connections


Sounds like fun, but as I say, a lot of work. Personally I have far too much to do right now to be able to seriously commit to a project like this, although if you want to do it, I will help where I can. Even just writing these posts is taking more time than I really should be spending on gaming.

There are a lot of things to consider when routing network flow. Here is an example of some of the basic issues: [Network Flow I] …. after that you can start factoring in some of the more technical bits.

Perhaps there is already an open source solution that would fit what we need for wc2, which could save the need to code the flow analysis/decision making from scratch, but even assuming that is is already done, just hooking this in along with the necessary support code to adapt the client to use it is a big job.

We must also remember that what we are always striving for is reliability and minimum latency, not maximum flow. Every extra step that is placed into the nework logic inheranty adds latency. A small amount of this to side-step disasterous connections between individual clients could add up to a net gain for the reliability of individual games, but too much of it and you're using a flame-thrower to get the weeds out of your lawn… the weeds are gone, but so is the lawn.

reroute if clients will suddenly change their IP during the game.
I think this would be a very rare event.

UDP over internet is far from perfect and has lots problems. So, some kind of layer between raw war2 UDP and internet (like distributed UDP network between war2 clients) based on new technologies and implementation is what we need.


UDP packects are inherantly no less likely to arrive at their destination than any other type. By rights, it *SHOULD* make absolutely no difference. There is a difference, but that is not due to any problem with the UDP spec, it does exactly what it was designed for. An explaination requires a bit of discussion about netowork data streams, so.......  Packets are arranged something like this like this:

This is a very simplified version just to illustrate the principles, there are other fields not listed here

Internet Layer:
Code: [Select]
An IP packet:

{version, size, protocol, src address, dst address, {Payload - 'size' bytes of data}}

Transport Layer:
Code: [Select]


UDP Packet:

       {src port, dst port, size, checksum, {Payload - 'size' bytes of data}}

TCP Packet:

      {src port, dst port, sequence number, size, checksum, {Payload - 'size' bytes of data}}


So at the internet layer, a UDP packet looks something like this:

Code: [Select]

{version, size1, 0x11, src address, dst address, {src port, dst port, size2, checksum, {Payload - 'size2' bytes of data}}}

Here size1 is the size of the entire UDP packet, which is the payload for the IP packet, and size2 is the size of the data which is the payload for the UDP packet.

... and a TCP packet would be something like:
Code: [Select]

{version, size1, 0x06, src address, dst address, {src port, dst port, X , size2, checksum, {Payload - 'size2' bytes of data}}}


So accoring to the original design, the IP packets are only being routed based on the information in the IP header which is pretty much just the address and the number of bytes. Its only when the packet is received by the network drivers at the other end that the IP headers are stripped and the payload is then passed on to the appropriate transport layer driver that the internal headers are inspected.

The IP header does, however contain a protocol field, which for these 2 cases will be set to either 0x06 for TCP or 0x11 for UDP, so servers along the way can see what type of data is in the payload and possibly decide to do something different with the packet.

Without an actual decision being made about it somewhere along the way, neither TCP, UDP or any other transport layer protocol is any more likely to be reliable than another. The main difference between TCP and UDP packets is the inclusion of the sequence number in TCP and the behaviour of the drivers handling transaction. TCP simply puts a number on each packet when it is sent, like the page number on a book.

So if the TCP driver receives packets 1,2,3,4,6,7,8 it sends back a request saying "hey I'm missing packet 5 - send that again!", and it blocks forwarding packets 6,7,8… etc. to the application until packet 5 is received and inserted into its correct spot in the data stream. This great for reliable data transfer, but terrible for anything that requires real-time transfer. This is why things like VoIP and live video streaming use UDP, because for them having the latest real-time data is more important than worrying about a tiny fraction of the data that was not received.

BUT…. and this is the big but… what if you are attemting to write firmware for a network router? What do you do when there is congestion in the network and you simply cannot transmit all of the data that you have received? Lets pretend that you are only getting TCP and UDP packets (indeed they make up the majority of the transport layer bandwidth). What if you drop a TCP packet? The driver at the other end is just going to send back a request for the missing packet, then the source is going to re-transmit the packet and you have just doubled your congestion problem.

On the other hand, most of the UDP traffic is stuff like VoIP and you know that a) losing 1 in every N packets in a VoIP stream doesn't stop it from functioning, it just gives a corresponding loss in signal quality and more importantly b) If you drop a UDP packet, most times this will just be accepted by the clients and you won't have to handle the traffic from them requesting and retransmitting the lost packet. Considering this, its no surprise that generally when a server is facing network conjestion issues the first thing they start doing is dropping UDP packets.

O.K. so lets just stamp all our packets with a 0x06 instead of an 0x11 right? So the servers will think the payload is TCP and be less likely to drop it… Nice in theory but there's a couple of issues with this. Firstly, the IP packet header is being written at the driver level, so it's not just a matter of altering wc2 client you would have to be hacking the network services on the machine it's running on (no thanks) and second, stateful and application layer firewalls are quite commonplace these days. These firewalls do inspect the payloads of internet, transport and even application layer packets looking for things like malicious activeX controls and the like. These would instantly flag incorrectly labeled IP packets and a the very least delay them if not block them all together.

ISPs also use a certain degree of this type of packet inspection to manage their bandwidth, for instance on Monday afternoons my torrenting speeds drop to about 10% of what I get the rest of the week. Torrents are distributed via anonymous peer-to-peer connections, and in theory my ISP should have not even know that the packets contain torrent data, but you can be very sure that they do.

These types of issues are why I suggested using a fast VPN  *COULD* be a way to improve a bad connection. Inherantly VPNs have a lot more overheads associated with them so should have higher latency that just sending UDP packets, however govornments, corporations and all sorts of high paying customers for the ISPs tend to use VPNs, and being secure, there is no way for anyone to have a look inside and know exactly whats there, as a result many ISPs tend to give high priority to this type of traffic, and they are the last type of packet that is likely to be vonluntarily dropped by a server.

…. and at this point, like so much that happens in our little community, our issues start to become less about network engineering and more about social engineering. ;)
« Last Edit: October 20, 2016, 09:27:02 AM by Lambchops »
its gooder to hax hard and NEVER get caught!

Offline Lambchops

  • Ogre Mage
  • ********
  • Posts: 1541
    • View Profile
Re: Server Lag : what it is and isn't....
« Reply #34 on: October 20, 2016, 05:11:17 AM »
As i see, a C++ class/library to include into war2 loader

Have you considered trying a later version of storm.dll?

I know that it was further developed for some years after bne was released, as was PvPGN, so provided that Blizzard kept it backward compatible (I'd hope so, they're generally pretty good) the latest compatible version of storm.dll might possibly be a pre-written 'drop-in' upgrade for the whole networking thing.

IDK what version CE uses already or if Blizzard made any improvements to the underlying networking code that would actually help, but as far as potential "bang for your buck" goes it might be worth checking out.... just a thought :)


its gooder to hax hard and NEVER get caught!