Warcraft II Forum

Warcraft II => Mods & Development => Topic started by: MaStA{hR} on October 31, 2016, 01:58:26 PM

Title: Server Update (reconnect to games when disced)
Post by: MaStA{hR} on October 31, 2016, 01:58:26 PM
Would it be possible to include this or would the coding not work for this game:

To reconnect to a game if you were disconnected?
Title: Re: Server Update
Post by: I hate naggers on October 31, 2016, 02:37:25 PM
couldn't this be archieved by save game?

We should direct our developer's attention (if we have any) on things that would largely impact our quality of life while being not so hard to implement (hardware ban hi)
Title: Re: Server Update
Post by: Lambchops on November 01, 2016, 06:09:42 AM
To reconnect to a game if you were disconnected?

This is a 100% client issue and not to do with the server. In fact THIS ALREADY HAPPENS.

This is what is happening when someone lags and nearly drops, but then their connection recovers. While everyone is looking at the lag screen, they are disconnected. Then they rejoin the game.

You really want laggers to be able to rj when they finally drop?

The issue is (again) that a wc2 game is not happening on a server, it's peer-to-peer. There is no 'official' version of the game state, instead all clients must agree to what is happening in the game for it to continue, if one client is out of sync they will drop.

To join a game in progress a client would have to negotiate with all the active clients in the game and come up with an agreed version of the game state, then they would all have to accept the joining client. At that point all sorts of exploits could come up, like people drop-hacking one player to replace them with somebody else who is spoofing them etc.

In the end, if the rest of the clients have decided to boot you, its for a reason - in most cases its either because your connection is too poor to allow the game to continue with you in it, or because your version of the game state is different to everyone else's.

Title: Re: Server Update
Post by: Lambchops on November 01, 2016, 06:26:52 AM
couldn't this be archieved by save game?

If someone wanted to do it, then yes, the save game mechanism is exactly what is needed. In effect you would be ending the game and saving it - which is then the 'official' version of the game state. Then you would modify saved game state to show that the disconnected  player is still in the game, then restart the game from the modded save. If you could automate the process it would in effect allow players to join a game in progress, but it could make the game suceptable to all sorts of exploits.
Title: Re: Server Update
Post by: tk[as] on November 01, 2016, 09:39:45 AM
I'm not sure if it works on all routers, it's probably isp related. . But when I lived in an apartment I used to get disconnected, drop screen would pop up.. then go for like 10 seconds, then disappear,  then pop back up for another 10 seconds until I eventually dropped.. I figured out I was able to tab out of war2, disconnect from my Internet connection, then auto reconnect and tab back into the game and keep playing without dropping
Title: Re: Server Update
Post by: Delete mine too on November 01, 2016, 11:26:41 AM
I'm not sure if it works on all routers, it's probably isp related. . But when I lived in an apartment I used to get disconnected, drop screen would pop up.. then go for like 10 seconds, then disappear,  then pop back up for another 10 seconds until I eventually dropped.. I figured out I was able to tab out of war2, disconnect from my Internet connection, then auto reconnect and tab back into the game and keep playing without dropping
This is why I made my lag defender, and the abuse of the screen shot key will cause the lag box. I can set the time to 0 and drop who ever is showing up without a waiting the 30 seconds.
Title: Re: Server Update
Post by: Lambchops on November 01, 2016, 02:24:46 PM
I'm not sure if it works on all routers, it's probably isp related. . But when I lived in an apartment I used to get disconnected, drop screen would pop up.. then go for like 10 seconds, then disappear,  then pop back up for another 10 seconds until I eventually dropped.. I figured out I was able to tab out of war2, disconnect from my Internet connection, then auto reconnect and tab back into the game and keep playing without dropping

Yeah that sounds like a busy ISP downgrading the priority of your connection, but when you reconnect you're back at the top if the list. It's pretty common practice. I'd assume they must have been assigning you a static address for you to be able to continue in the same game?
Title: Re: Server Update
Post by: iL on November 01, 2016, 04:41:12 PM
couldn't this be archieved by save game?

We should direct our developer's attention (if we have any) on things that would largely impact our quality of life while being not so hard to implement (hardware ban hi)
Well, some kind of autosave could be a key. Also, autosavr could be a separate project not related to others (like hardware ban), so it can be implemented by somebody who wants (and can) to help.
Another question is if that is really useful: even if we have saved game, we have to join all the previous players together again. Not so easy task to make them do that.

Then you would modify saved game state to show that the disconnected  player is still in the game, then restart the game from the modded save. If you could automate the process it would in effect allow players to join a game in progress, but it could make the game suceptable to all sorts of exploits.
I'd say just to make saves periodically, considering disconnects. To let hoster know which save-file should be reopened. Game load could be a manual process.

Yeah that sounds like a busy ISP downgrading the priority of your connection, but when you reconnect you're back at the top if the list. It's pretty common practice. I'd assume they must have been assigning you a static address for you to be able to continue in the same game?
Yes, i never saw such behavior, but could be quite possible: if the connection restores, traffic can just continue to go. Thanks to UDP protocol again.
Also, if there's an external connection (router to internet) restarting, your external ip should be the same to let it work.
Not sure about breaking and restoring internal connetion between router and client: is it possible to restore the connection if your internal ip has been changed? I mean, disc, ip changed from 192.168.0.2 to 192.168.0.3 and then your connection to war2 being restored. I think even this behavior is possible.
Title: Re: Server Update
Post by: tk[as] on November 01, 2016, 05:09:45 PM
Static IP. wireless net gear router. IPv4 address stays the same
Title: Re: Server Update
Post by: EviL~Ryu on November 01, 2016, 08:33:42 PM
Static IP. wireless net gear router. IPv4 address stays the same


This.


Sent from my Nokia 3310 using Tapatalk
Title: Re: Server Update
Post by: Lambchops on November 02, 2016, 05:24:08 AM
Static IP. wireless net gear router. IPv4 address stays the same

I meant the external external address your ISP is assigning to you, that is what the other clients have.

Are you talking about reconnecting to your router or reconnecting to your ISP? Once the game is started changing your LAN address probably wouldn't be a problem, but changing the public address that your ISP is assigning to you would be.

If you were just talking about reconnecting to your router then probably its not your ISP, its a settings/firmware issue with your router.
Title: Re: Server Update
Post by: Lambchops on November 02, 2016, 08:47:49 AM
Quote from: CLAW on October 31, 2016, 02:37:25 PM

    couldn't this be archieved by save game?

    We should direct our developer's attention (if we have any) on things that would largely impact our quality of life while being not so hard to implement (hardware ban hi)

Well, some kind of autosave could be a key. Also, autosavr could be a separate project not related to others (like hardware ban), so it can be implemented by somebody who wants (and can) to help.
Another question is if that is really useful: even if we have saved game, we have to join all the previous players together again. Not so easy task to make them do that.

Quote from: Lambchops on November 01, 2016, 06:26:52 AM

    Then you would modify saved game state to show that the disconnected  player is still in the game, then restart the game from the modded save. If you could automate the process it would in effect allow players to join a game in progress, but it could make the game suceptable to all sorts of exploits.

I'd say just to make saves periodically, considering disconnects. To let hoster know which save-file should be reopened. Game load could be a manual process.

To be honest, I've never used the save game feature for a multiplayer game, but modding/automating features that already exist in the game is much easier that trying to introduce entirely new ones.

I've always assumed the save game feature makes a file based on the existing game state with info about the players and the current state/actions of each unit - then you just start a new game based on a combination of that file and the original PUD.

Just did a few tests on my own and it seems pretty good at restoring the game state. i.e. spell effects/corpses/current actions etc. were all resumed.

With several players in the game i assume it only lets the same players join the restored game? I know the dialog says that only the host can save the game so I assume it is only saved to their computer, so when it is restarted the save game file has to be downloaded by the rest of the clients?

I don't imagine autosaving would be terribly difficult. Certainly it would be trivial with a macro, but that would be intrusive to an active player. Periodically calling the save game proc with an auto-generated file name (autosaveXXX.mlt) shouldn't be too hard though.

------------------------------

*In theory* allowing a player to join a game in progress could be achieved by something like this:

Trigger the clients with a pre-defined in game messages... with some sort checksum based on the stuff about the current game.

So the player wanting to join messages a player in the game /m player #join

The clients have a bit of code injected into them so if they detect incoming messages with a '#' prefix so if they get a "#join" message incoming they generate an in-game message that all clients see like "#request:JoiningPlayer:945B98D1"

The #request message triggers the host to message the joining player details game datails like "#invite:111.222.123.2:6112:945B98D1"

The joining client then replys with "#confirm:join:945B98D1"

When the host gets time they generate an in-game message like "#restart:111.222.123.2:6112:945B98D1"

The rest of the in-game clients reply with "#confirm:restart:945B98D1"

When all clients have confirmed, the host save the game then sends "#execute:945B98D1" both to the joining player and in game.

All clients then leave the game.

The host client mods the save game file to include the new player, and creates a new game from the save, while the non-host clients join the created game, along with the joining player.

The host would wait for all players to join the new game then start it and the game would continue.

Ideally the new game would be created and players would join without using the server game list mechanism, the server would only be used to relay message to/from the joining player.

---------------------------

This is, however, a lot of work for what is debatably little benefit.

What is the use of it? For a player to rejoin if they lag out? If they are lagging they will probably still lag when they rejoin, and if everyone agrees this can already happen with the existing game save if they do it before the player actually drops.

If the joining player wasn't already in the game what do they start with and where? 1 peon at an unused starting location? What if an enemy already has an exp there? How are they going to catch up anyway? Really, what's the point?

I suppose it might be nice for people to be able to join as watchers, but is that really worth it? What if a lagging watcher joins a game? What if people keep joining and leaving causing annoying pauses in the game?

The .MLT files are much bigger than .PUD files, and the game's download procedure is VERY slow... The biggest .PUD files are around 120KB. Using normal network protocols, this should take less than 1 second to download even on a slow connection, but as we all know whatever method the game client uses is MUCH slower than this. My test .MLT file it 530KB so it would seem using only existing features a game restart is going to take maybe 2-5 minutes (yuk).

The other thing about this mechanism is is that there is a massive security issue, in that while the game is being restarted, a single client has the one official version of the game state, so at that moment they can play God and change anything at all in the game.

This potential exploit, however has some interesting possibilities for creating RPG style campaigns and the like... i.e. when set of triggers in the game is satisfied, the game can be automatically restarted with any changes the designer wanted... introducing new cpu enemies, allowing certian units/abilities that were previously restricted. It's a nice concept, but for the probably thousands of hours it would take to impliment, at that point you may as well go play wc3.

-----------------------

For my money, there is one WC2 mod that is head and shoulders above everything else absolutely screaming out to be implimented. (see next post :) )

Title: Client Update!
Post by: Lambchops on November 02, 2016, 09:50:22 AM
there is one WC2 mod that is head and shoulders above everything else absolutely screaming out to be implimented.

That mod is breaking the 128x128 map size limit. Historically I know this has been looked at before, but I dont see why it couldn't be done.

It also leads to another 2 mods that I would like to see, and they are :
 - Increasing the game display from the rather miserly 14x14 window we currently have
 - Breaking the maximum per game unit cap - cannot create unit - this one could possibly be quite easy.

Internally locations are mostly stored as 2 WORD size variables and often passed or referenced as a single DWORD argument so most of the game code already supports a map size of up to 65536x65536. There are, however parts of the code that impliment WORD sized absolute pixel co-ordinates, which restricts things to 2048x2048, but even this would be pretty cool.

Implimentation would not be trivial, but it could be almost all done just by modifying the existing code without actually having to add any new code in.

We would have to do this (at least!)

- Mod existing PUD editors to allow the creation of large maps.

- redefine the valid .PUD check so the clients will accept larger maps

- Mod things like any internal co-ordinate checks that may declare co-ords >127 to be invalid, it would be nice if these would all be based on the map size in the PUD file, but as there are currnetly only 4 legal map sizes there will probably be sections that are hard coded to make calculations based on one of these 4 sizes. All this code would ned to be chased down and modded or re-written.

- Increase any buffer allocations to allow for bigger maps, possibly parts of the executable's .data and .data? sections will also have to be redefined to allow more space.

- Check and mod the display code to allow for larger maps. Scaling for the mini-map is an obvious one, as is the buffer that keeps the background updated (this one is why you can see where trees are chopped even if you don't currently have vision there).

The main view code probably wouldn't have to be changed at all apart from any checks that just flat-out reject large co-ords.

.... Possibly by this time we might know enough about the display code to increase the 14x14 window size by having the internal routines draw the units to a larger buffer, then resizing that image and pasting it onto the existing screen buffer. There are also a bag of issues related to translating mouse input from this resized window.

--> This also leads to running the game at higher screen resolutions, which would be nice, but that would be another BIG mod. Where do you draw the line? It would be nice for it to actually be done before we are all too senile to enjoy it ;)

Breaking the unit cap should just be a matter of increasing the buffer allocation and finding any bits of code that impliment:

if n_units>MAX_UNITS {say "BLEH!"}







Title: Re: Server Update
Post by: Delete mine too on November 02, 2016, 02:27:15 PM
The most I've gotten to is war2 sitting at black screen when I used my maps bigger then 128x128. I think I nulled out the minimap and it no longer crashed. But I was stuck there and clueless. What do you think about that?
Title: Re: Server Update
Post by: Lambchops on November 02, 2016, 04:21:03 PM
The most I've gotten to is war2 sitting at black screen when I used my maps bigger then 128x128. I think I nulled out the minimap and it no longer crashed. But I was stuck there and clueless. What do you think about that?

I think you gave up to soon ;)
Title: Re: Server Update
Post by: EviL~Ryu on November 02, 2016, 04:30:01 PM
The most I've gotten to is war2 sitting at black screen when I used my maps bigger then 128x128. I think I nulled out the minimap and it no longer crashed. But I was stuck there and clueless. What do you think about that?

I think you gave up to soon ;)

Continue his work Lambs! We all believe in you!!


Sent from my Nokia 3310 using Tapatalk
Title: Re: Server Update
Post by: Delete mine too on November 02, 2016, 05:40:42 PM
The most I've gotten to is war2 sitting at black screen when I used my maps bigger then 128x128. I think I nulled out the minimap and it no longer crashed. But I was stuck there and clueless. What do you think about that?

I think you gave up to soon ;)
Nah I just ran out of time because of career changes. Plus tons of other projects kept me bottlenecked. All honestly I was tired of being the one trying.
Title: Re: Server Update
Post by: shesycompany on November 02, 2016, 07:35:57 PM
hell naw,just gonna have to charge back up!we got war2 mini also ill see if i can finish it up.
Title: Re: Server Update
Post by: Delete mine too on November 02, 2016, 08:02:44 PM
hell naw,just gonna have to charge back up!we got war2 mini also ill see if i can finish it up.
It's been months now for me around 6 months I say. I'm trying to get a feeling again but I think it finally was murdered by other fellow admins. Unless a handful of people retire or get released from their admin status I doubt I will return my efforts.

Soon ru will be another war2 site you have to see from the waybackmachine. Activity is just dying and what's left is being ran right into the ground. I suggest you all jump off before the ship crashes. I wish you all luck!



Title: Re: Server Update
Post by: MaStA{hR} on November 03, 2016, 02:41:06 AM
hell naw,just gonna have to charge back up!we got war2 mini also ill see if i can finish it up.
It's been months now for me around 6 months I say. I'm trying to get a feeling again but I think it finally was murdered by other fellow admins. Unless a handful of people retire or get released from their admin status I doubt I will return my efforts.

Soon ru will be another war2 site you have to see from the waybackmachine. Activity is just dying and what's left is being ran right into the ground. I suggest you all jump off before the ship crashes. I wish you all luck!





Da fuq bro? I admit we have subpar admin overrall. But membership averages is actually up recently. I don't know if it's that time of year or what. With that being said, status quo really sucks, because 25-75 people online is not a lot at all. But hey, it's not dead yet. :) It's the same as it was when I first joined ru in like 2010.
Title: Re: Server Update
Post by: Lambchops on November 03, 2016, 11:43:53 AM
The most I've gotten to is war2 sitting at black screen when I used my maps bigger then 128x128. I think I nulled out the minimap and it no longer crashed. But I was stuck there and clueless. What do you think about that?


I think you gave up to soon ;)

Nah I just ran out of time because of career changes. Plus tons of other projects kept me bottlenecked. All honestly I was tired of being the one trying.


Fair call mate. This I understand.

Having slept on it, I must also say thanks for bit of actual real info. Getting any actual info about how the client works that comes from somebody else's time and effort is always good. Nobody understands better than me that your 'blank screen' story was no doubt the result of many, many hours of hard work.

So here's some thoughts about where to go next:

There are really only 2 questions for this type of thing:

1) what is it doing?
2) how do I make it do what I want?

They inevitably lead back into each other. Working out a bit of 'What is it doing?'(1) leads you to do/mod something (2) which almost never has the effect you thought it would so you go back to (1) and work out what the actual effect was, then try something else(2) etc...

(1) can be a bitch. (2) is the fun part - coming up with elaborate schemes muhahahahahah... ;)

! No longer available (http://www.youtube.com/watch?v=pwlCzYzjYls#)


So, as tends to be the case, you are stuck at (1), however a blank screen should not be a deterrant. The game screen is the least likely place to get any real info about what the internals are doing when you are messing around with them. It generally returns only a boolean 1-working, 0-fucked up.

So what is it doing? If it is not raising any exceptions with the OS then at a guess I would think that either (a) it's stuck in a loop somewhere, or (b) 1 or more buffers have overflowed and corrupted .data or .data? section values related to the video and sound.... or possibly (a) as a result of (b)

You need to have a look at the running process. Belt this stuff around a bit ;)

    MASM32 (http://www.masm32.com/)
    ReadProcessMemory (https://msdn.microsoft.com/en-us/library/windows/desktop/ms680553%28v=vs.85%29.aspx)
    OllyDbg (http://www.ollydbg.de/)
    IDA Pro (https://www.hex-rays.com/products/ida/support/download_freeware.shtml)

Your're welcome. :)

... being ran right into the ground. I suggest you all jump off before the ship ...

awww buck up little camper :) Sorry I wrote most of this post this morning but I got busy and it took me all day to get back to it....
Title: Re: Server Update
Post by: shesycompany on November 03, 2016, 12:04:17 PM
he should of done onscreen like a pro ::) its actually easier to go smaller than bigger ;D...side story stayed at my grandads place over the weekend..had my lappy and a usb for all the war2 stuff and have lost it :o so all halt right now for me also :'(..most of it was piles of notes that i no longer need anyhow.
Title: Re: Server Update
Post by: Delete mine too on November 03, 2016, 12:43:46 PM
The most I've gotten to is war2 sitting at black screen when I used my maps bigger then 128x128. I think I nulled out the minimap and it no longer crashed. But I was stuck there and clueless. What do you think about that?


I think you gave up to soon ;)

Nah I just ran out of time because of career changes. Plus tons of other projects kept me bottlenecked. All honestly I was tired of being the one trying.


Fair call mate. This I understand.

Having slept on it, I must also say thanks for bit of actual real info. Getting any actual info about how the client works that comes from somebody else's time and effort is always good. Nobody understands better than me that your 'blank screen' story was no doubt the result of many, many hours of hard work.

So here's some thoughts about where to go next:

There are really only 2 questions for this type of thing:

1) what is it doing?
2) how do I make it do what I want?

They inevitably lead back into each other. Working out a bit of 'What is it doing?'(1) leads you to do/mod something (2) which almost never has the effect you thought it would so you go back to (1) and work out what the actual effect was, then try something else(2) etc...

(1) can be a bitch. (2) is the fun part - coming up with elaborate schemes muhahahahahah... ;)

! No longer available ([url]http://www.youtube.com/watch?v=pwlCzYzjYls#[/url])


So, as tends to be the case, you are stuck at (1), however a blank screen should not be a deterrant. The game screen is the least likely place to get any real info about what the internals are doing when you are messing around with them. It generally returns only a boolean 1-working, 0-fucked up.

So what is it doing? If it is not raising any exceptions with the OS then at a guess I would think that either (a) it's stuck in a loop somewhere, or (b) 1 or more buffers have overflowed and corrupted .data or .data? section values related to the video and sound.... or possibly (a) as a result of (b)

You need to have a look at the running process. Belt this stuff around a bit ;)

    MASM32 ([url]http://www.masm32.com/[/url])
    ReadProcessMemory ([url]https://msdn.microsoft.com/en-us/library/windows/desktop/ms680553%28v=vs.85%29.aspx[/url])
    OllyDbg ([url]http://www.ollydbg.de/[/url])
    IDA Pro ([url]https://www.hex-rays.com/products/ida/support/download_freeware.shtml[/url])

Your're welcome. :)

... being ran right into the ground. I suggest you all jump off before the ship ...

awww buck up little camper :) Sorry I wrote most of this post this morning but I got busy and it took me all day to get back to it....


Okay I'll have a deeper look at it. Hopefully I can set aside some time this weekend ;)

Tip

That .data file can be opened and viewed! Easycompany showed me that!
Title: Re: Server Update
Post by: shesycompany on November 03, 2016, 12:53:34 PM
it may be possible without crashing man gl!ive not hit any mini map values,besides the gold mine sending me and the cpu to a part of the map lol...ill go back and look and see if i can hopefully take the mini out.
Title: Re: Server Update
Post by: Lambchops on November 03, 2016, 01:20:26 PM
he should of done onscreen like a pro ::) its actually easier to go smaller than bigger ;D...side story stayed at my grandads place over the weekend..had my lappy and a usb for all the war2 stuff and have lost it :o so all halt right now for me also :'(..most of it was piles of notes that i no longer need anyhow.

..... oh, there was an address here before that I was going to check out for you, but it has vanished, if you still need repost or let me know.

Title: Re: Server Update
Post by: shesycompany on November 03, 2016, 01:35:09 PM
i found your old post, just the way people write addresses.
i was checking dos war2.
Title: Re: Server Update
Post by: Lambchops on November 03, 2016, 06:43:46 PM
That .data file can be opened and viewed

Cheers.  :) I know what you mean there. Also....



PE files (https://en.wikipedia.org/wiki/Portable_Executable) contain sections.


Common sections are:

.text (AKA .code) which contains the compiled ML (https://en.wikipedia.org/wiki/Machine_code)

.data   which contains variables and strings etc., that the program can use and modify as it runs

.data? is the same as .data, except that it initially contains no values (uninitialised data). The .data? section is a virtual section that has 0 size in an .exe file but is created when the image is mapped to a process' memory by the system loader. It's a way of pre-defining memory to be allocated, with the bonuse of that memory being inside the module's image.

.rsrc "resource" usually contains things like bitmaps, constant strings (that are not modified during execution), dialog templates etc...



Not all files have all sections, and indeed the names, despite being almost always as listed above, are just a convention. The actual function of a given section is defined by flags set in the Characteristics member of the IMAGE_SECTION_HEADER structure.

Everything you never wanted to know about windows .exe files is here (http://www.microsoft.com/whdc/system/platform/firmware/PECOFF.mspx). (You want the file called "pecoff.docx")

'COFF' stands for Common Object File Format. Executable files started out as a sub-set of COFF files, they share many internal structures. Things like .obj "object" files are COFF files - .obj files are produced by compilers which are then fed to linkers which create the finished .exe file. 


.... anyway so what I mean by this:
....buffers have overflowed and corrupted .data or .data? section values...

      is that when the image is mapped to memory certain areas will have been allowed for certain data, that has an expected maximum size, so there could, for instance be something like this:

Code: [Select]
.data?
mapArray   db 4000h dup (?) ;-- some map array or other
hGLthing   dd ?             ;-- handle to some important GL structure
otherStuff dd ?,?,?,?       ;-- all sorts of other important data

as the max map size is 128x128 = 16384 (0x4000) and the map size is checked when it is loaded, whatever routine that puts data from the map or whatever into that array doesn't bother doing any checking, so when we gymp the 128x128 map size checks, the program just goes right ahead and writes data that is supposed to all fit in that array straight over the top of all the variables that are stored after it.

This is where image data sections can be tricky, because all the references to them are hard coded into the ML instructions. From one angle this makes them very easy to debug because you can always immediately see what is being accessed by a given instruction.

When memory is dynamically allocated, its trickier to debug, but much easier to resize. the same thing could be achieved with:

Code: [Select]
.data?
ptrMapArray   dd ?           ;-- *POINTER* to some map array or other
hGLthing      dd ?           ;-- handle to some important GL structure
otherStuff    dd ?,?,?,?     ;-- all sorts of other important data

then somewhere in the initialization code there is:
Code: [Select]
.code
push 4000h
call GlobalAlloc
mov ptrMapArray,eax     ;  <-- functions return values in the eax register
so this just asks the OS to allocate space for the array then just stores a pointer to the allocated memory in the data section. Later when the program wants to access this array you might see something like:
Code: [Select]
mov edi,ptrMapArray
; ...... then further on
mov [edi+ebx],dl ; ... or whatever
in this case edi woud contain the pointer to the base address of the array, ebx would contain an offset into the array, and the low byte of edx would contain the value it is writing there.

At least that is what the source could look like. Once it is compiled then decompiled, you dont have the benefit of the meaningful variable names that the guy coding it in the first place used, so your:
Code: [Select]
mov edi,ptrMapArray  has become maybe:
Code: [Select]
mov edi,[4A789Ch]... where 0x4A789C is the actual address of that dword in the .data? section (this is what the linker does)

things like:
Code: [Select]
call GlobalAllocwill still be there because GlobalAlloc is exported by name from kernel32.dll and the disassembler will extract the proc name from the IMAGE_EXPORT_DIRECTORY structure in kernel32.

... so if our array is dynamically allocated, and we want to change the max map size from 128x128 to 256x256 all we need to do to resize this particular buffer is change:

Code: [Select]
push 4000h    ; (128x128)
call GlobalAlloc
to:
Code: [Select]
push 10000h    ; (256x256)
call GlobalAlloc

BUT....
....  if this buffer (and not a pointer to it) was itself allocated in the .data? section, to resize it we have to create more space in that section of the PE image.

This in itself is trivial - you just adjust the VirtualSize member of the IMAGE_SECTION_HEADER structure for the .data? section and the system loader will create a larger image - however this doesn't change the ML in the .text section, and as I mentioned the addresses of the variables, and importantly for this case, those after the array (i.e. 'hGLthing') have been hard-coded into the ML instructions by the linker, so the code is still going to read and write the value of that handle to the same location, and the code writing to the array is still going to corrupt it. In this case all we have done is allocate extra memory at the end of the .data? section that isn't used anyway.

To resize this kind of array you would have to increase the size of the .data? section by N bytes, then find every reference to anything in the .data? section that is above the base address of that array (there could be hundreds of addressed data locations with multiple reference to each) and increase all of these addresses, which are hard-coded into the instructions, by N bytes, and to make matters worse there could be pointer tables to some of these locations stored literally anywhere that would all need to be found and adjusted.

With the original source code, you just change your:
Code: [Select]
mapArray   db 4000h dup (?)     to:
Code: [Select]
mapArray   db 10000h dup (?)     then recompile and its done (approx. 5 seconds total)

WHen reverse engineering a complex program this is can be almost impossible, and WC2 was pushing the limits of CPU and RAM capacities back in 1996, so to get the maximum possible performance they used lots of little tricks involving pointer tables to reference things and also to control program flow. It really is a cool chunk of code (Lamby gets all dreamy-eyed and cracks a nerd-boner) but it can be hard to get your head around.

....anyway if faced with this situation I would opt for injecting a small amount of code somewhere that dynamically allocated a larger buffer, and just leave the buffer in the .data? section as a blank space. Then every instruction that accessed the original buffer would have to be located and modded to point to the dynamically allocated one, which could be a bit painful, but nowhere near as painful as doing this for every one of the possibly hundreds of memory locations in the data section that are above the address of the original buffer.... and a nice trick for this type of thing is using a debugger to put a memory-watch on the original buffer then pause execution and report the value of the EIP register whenever it is accessed ;)

And for specifically Blizz related allocation, much of the WC2 stuff is, as you probably know, stored using Mike O'Brians Mo'PaQ format (.mpq files on disk). These are loaded into dynamically allocate memory by various functions exported by ordinal from storm.dll, so if any of these need to be resized it would be done by adjusting the size of the relevent resource in the .mpq archive.


Some notes on the code bits here:
It's all ASM code because that's how I think when I'm doing this stuff, and it really is the only way to think when you're reverse-engineering anything.

DATA DECLARATIONS

Code: [Select]
.data?
hGLthing   dd ?
this just allocates 4 bytes in the .data? section and names that location "hGLthing "
'dd' is 'declare double' it allocates a DWORD
'dw' allocates a WORD (2 bytes)
'db' allocates a BYTE
(also 'dq' allocates a QWORD, but you won't find any of those in WC2)

with this:
Code: [Select]
mapArray   db 4000h dup (?)'dup' means duplicate. Its declaring bytes (db) and its declaring 0x4000 of them.
In all of these the '?' is the data value the location is initialised with.
Because these are in the .data? section which holds only uninitialised data they can't have any defined initial value do they must all be '?'.

In the .data section you can have initialised data so you might see:
Code: [Select]
.data
dwThisVar   dd 0FE339D2h
myByte   db 20h
thisString db "Hello World",0   ;  <-- an initialised zero terminated string
myWords dw 3,3,3,3     ; <-- 4 words all set to 3 (8 bytes all up)
ymWords dw 4 dup (3)   ; <-- exactly the same ( 4 words all set to 3 )

when compiled this .data would look like:
Code: [Select]
D2:39:E3:0F:
20:
48:65:6C:6C:6F:20:57:6F:72:6C:64:00
03:00:03:00:03:00:03:00:
03:00:03:00:03:00:03:00:
(off the to of my head - dont crucify me if I messed up, but you get the idea..)

WC2, was of course written in C, but it all compiles to ML anyway. Apart from ASM (which is for the most part 1:1 ML instructions) C is the lowest level language around and therefore closest related to ASM and the compiled .exe, also most C compilers support in-line ASM code, so performance critical sections of code can be written in ASM, within the C source and then all compiled at once.

Data containers declared in C end up roughly the same as they would have if declare in ASM, i.e. a C pointer variable '*ptrArray' or whatever, if declared globally would end up as a 'ptrArray dd ?' in the data section, whereas if it was declared locally in a procedure it would be created as a stack variable. Which is temporary use of the 'stack' to store values. The stack is a special area of memory that is heavily used by the CPU to shunt numbers around... the 'push' and 'pop' instructions move values onto and off the stack. It acts like a big pile of numbers (FILO) but the current stack location is held in the ESP register ... the stack pointer.

.. and OMG don't look now, but here here comes a bit of compilers, procs and stack frames
  :o

This leads to the actual compiler that was used to compile the source, as they have diferent methods for implimentng procedure calls and allocating local storage. You can allocate 4 bytes of stack space simply with:
Code: [Select]
sub esp,4but it is vital to ensure that the stack is 'balanced' - returned to its correct state before executing the next 'ret' - return instruction. In this case you could
Code: [Select]
add esp,4or just
Code: [Select]
pop eaxwhich would get the value off the stack and put it in the eax register. (push and pop automatically adjust the esp register). The other way to do this is to save the value of esp when you enter a procedure.

The Microsoft compiler does this by repurposing ebp - the base pointer to save the value in esp when entering a procedure, so it might produce something like:
Code: [Select]
push ebp   ; save the value in esp
mov ebp,esp ; use ebp to store the current stack pointer
sub esp,8  ; in this case 8 bytes of local storage has been allocated for the proc (most likely 2 DWORDS)
At this point the arguments that were passed to the procedure are also sitting on the stack behind the return address that was left there by the call instruction that called the proc and the original value of ebp, which we have just stored there. ebp holds the value of the stack pointer after this proc entry, so if it is not changed the passed arguments can be accessed directly using ebp provided we allow for the 8 bytes that these 2 DWORD values use, so the first argument can be accessed at [ebp+8] and assuming its a DWORD (which they almost always are) the second argument is at [ebp+0Ch] etc. At the same time the locals that were allocated with the
Code: [Select]
sub esp,8can be accessed at [ebp-4] and [ebp-8]. This area of the stack, from the start of the passed arguments to the end of any local allocations, for the life of the proc is often referred to as the stack frame for that procedure.

Meanwhile you can merrily push and pop your heart out or call other procs etc. (sending the stack pointer all over the shop) because the position of the stack pointer at the proc entry is stored in ebp, and the entry value of ebp is itself stored on the stack. So when it comes time to exit the proc they just do the following:
Code: [Select]
mov esp,ebp ; <-- restore the stack pointer from the base pointer
pop ebp ; <-- pop the base pointer value off the stack
ret         ; <-- pop the return address off the stack and jmp to that location
In actual fact, Mr Gates made such extensive use of this method that his mates at Intel eventually designed their CPUs with special instructions to acommodate it. The leave instruction does the combination of mov esp,ebp and pop ebp in one hit.

Anybody who has not nodded off by now may have noticed the other issue. Despite having balanced the stack from procedure entry to exit (thus taking care of any locals allocated), whatever arguments were passed to the proc when it was call are still sitting on the stack, as these were pushed there by the caller prior to calling the proc, so esp still has to be increased by an amount equal to the total size of the passed args to put things to rights, but this has to happen after the return value is removed from the stack. Alternate versions of the ret instruction (often denoted as retn) that take a numeric value are used for this, so at the end of a proc that is passed 2 DWORD args, you might see:
Code: [Select]
mov esp,ebp ; <-- restore the stack pointer from the base pointer
pop ebp ; <-- pop the base pointer value off the stack
retn  8     ; <-- pop the return address off the stack, add 8 to esp, then jmp to that address
OR...
Code: [Select]
leave <-- do the esp/ebp square dance
retn  8     ; <-- pop the return address off the stack, add 8 to esp, then jmp to that address

...sadly, however. Blizzard didn't use MASM to compile WC2. By the looks of it, it's Borland code,....

<EDIT>
...Actually, on further investigation, it looks like they used M$ VisualC 2.0 (https://en.wikipedia.org/wiki/Visual_C%2B%2B#Strictly_32-bit_versions)  Presumably there's compiler options for proc layouts/calling conventions etc. Interesting, but changes nothing anyway - it is what it is ;)
</EDIT>


.... not that there's anything wrong with that as far as executing it goes, but when decompiled, the stack frame addressing isn't anywhere near as clean to read.

For example, where the MASM code references arg1 by [ebp+8], and anywhere in that proc that you see [ebp+8] you know that it is addressing arg1, Borland doesn't bother with using ebp and just adresses arg1 as [esp+4]

..... until the first time a push instruction is used then 4 is added to esp, so now arg1 is at [esp+8] ... push again its at [esp+0Ch]... then pop something and its back to [esp+8] etc.

So the location of the first argument to a proc is esp + (number of pushes - number of pops +1) x 4 from the proc's entry point.

    EDIT: The exception is, of course, values that are pushed onto the stack as
    arguments to another proc that is being called from within the proc being
    examined, these will be balanced at the exit of the proc being called.


Modern deompilers like IDA (btw use that link to the free version 5 - they don't advertise that, and all the other versions cost mega$) take a fair bit of the hard work out of this for you by getting creative about how they present their disassembly, but due to the way WC2 uses embedded proc call addresses for optimisation you will still see red flags all over the place warning about stack imbalances because there's simply no way for the decompiler to decipher where many of the procs are being called from, so no way for them to be able to interpret what the state of the stack is before the call.




OK .... lol

I know this is a shit-ton of geekage, so I don't expect you to absorb it all at once, however I wouldn't have wasted my time if I didn't think you (and a couple of others who I expect are following...) were up to it. So come back to this post and re-read it occasionally because DAMN I wish somebody had sat me down and explained even half of this stuff to me 20 years ago.....
I love you assholes, really I do.  :fro:
Title: Re: Server Update
Post by: Delete mine too on November 03, 2016, 08:25:33 PM
Holy shit!!!!! Where have you been dude? Yes I will be back now just need to time to hide my nerd boner now!

LAMB FOR ADMIN!!!!!!
Title: Re: Server Update
Post by: tk[as] on November 04, 2016, 07:55:37 AM
Who is lamb btw??  Lance? .. Dude comes out of nowhere and knows everything server/war2 related
Title: Re: Server Update
Post by: I hate naggers on November 04, 2016, 08:35:52 AM
he was a chop player but i owned his ass
Title: Re: Server Update
Post by: Delete mine too on November 04, 2016, 09:37:36 AM
Who is lamb btw??  Lance? .. Dude comes out of nowhere and knows everything server/war2 related

I'm thinking he came from starcraft or something like that he sounds very professional. It would not suprise me if he is an old blizzard employee!
Title: Re: Server Update
Post by: EviL~Ryu on November 04, 2016, 10:30:32 AM
Who is lamb btw??  Lance? .. Dude comes out of nowhere and knows everything server/war2 related

I'm thinking he came from starcraft or something like that he sounds very professional. It would not suprise me if he is an old blizzard employee!

Wouldn't blame him, blizzard pay sucks.


Sent from my Nokia 3310 using Tapatalk
Title: Re: Server Update
Post by: shesycompany on November 04, 2016, 10:35:27 AM
cool stat banner!

lambs may just be a nerd :o

http://www.geek.com/games/hidden-river-city-ransom-password-discovered-after-27-years-1675405/ (http://www.geek.com/games/hidden-river-city-ransom-password-discovered-after-27-years-1675405/)

take nerds corner and put it in mods?its getting burried
Title: Re: Server Update
Post by: Delete mine too on November 04, 2016, 12:52:02 PM
Lamb post all your beautiful info in the mods section. In the welcome thread! That's what that thread is for!
Title: Re: Server Update
Post by: Lambchops on November 04, 2016, 04:34:06 PM
Who is lamb btw??  Lance? .. Dude comes out of nowhere and knows everything server/war2 related

I'm thinking he came from starcraft or something like that he sounds very professional. It would not suprise me if he is an old blizzard employee!

Wouldn't blame him, blizzard pay sucks.


Sent from my Nokia 3310 using Tapatalk

sadly no lol

just a random nerd.

and I've been kicking round this server for long enough.
Had a few years off, but I've been back for about a year now.

My ex is currently in the process of screwing me out of my house. Not sure where I'm going to go when she does, so if I suddenly disappear, that's what happened, but I'll be back.

Lamb post all your beautiful info in the mods section. In the welcome thread! That's what that thread is for!

Ok.

Also, if one of the mods feels like moving all that sort of stuff to one place, thats cool with me.





Title: Re: Server Update
Post by: Lambchops on November 04, 2016, 04:43:00 PM
he was a chop player but i owned his ass

lol, yes suitably glib. But you have more to contribute than just your sense of humor ;)
Title: Re: Server Update
Post by: mousEtopher on November 04, 2016, 04:58:46 PM
Should this whole thread go to mods? It's 90% lamby stuff right?
Title: Re: Server Update
Post by: shesycompany on November 04, 2016, 05:20:43 PM
yeah