Samstag, 10. Dezember 2011

Regarding compressed communication

This blog has been inactive for quite a while, but users where still commenting. I didn't have the time to respond, but I kept watching the progress. Now is the time to sum up some of the comments.

Recently the idea came up if the communication between the client and the server might be compressed (additionally to being encrypted). I don't think this is the case. From my understanding the game does most likely use opcodes for data exchange. The amount of data is not very high so having the cpu overhead of doing stream encrypting does not really bring much benefit.

Our commenters have also tested to use a "verbose" version of the zlib that writes the protocol of the called method. From what we could observe only calls to "decompress" were made. Therefore it's most likely that the communication is only RC4 encrypted data.

I also was contacted by an Austrian developer that is looking into doing the same attempt we are doing for DDO. He is using easyhook (http://easyhook.codeplex.com/) to trace the calls. He focused on CryptImportKey() and CryptGenKey() to receive the private/public keys and the RC4 stream cipher. This might be an option for Windows users to figure out what the client is doing (Linux users can always use wine and hack it in whatever way they prefer).

I'm looking forward to what else our readers might come up with. Feel free to comment on this blog for whatever reason you like, e.g. connecting to other developers.

14 Kommentare:

  1. DDO is not using CryptEncrypt() any more after the first welcome packet has been encrypted with the public key of the server. So if the stream really is encrypted with RC4, it has to use the RC4 key in its own RC4 implementation. What benefit that brings over using the already initialized RC4 from Windows eludes me. Or DDO is simply using some kind of library for RSA, that automatically generates a RC4 key just in case the user of the library decides he wants to use an RC4 cipher key.

    Even if you are applying the RC4 key, you extract from DDO to decrypt the packets, you have little way to verify that what you decrypt is actually the plain text. Although there must be some human understandable plain text around: Character Names. Since it's reasonable to assume that character names are sent from the server to the client they *must* be in the packets, and the decryption can be verified once you find them in your plain text. Finding the character data is actually quite simple: The huge packets at the beginning when the client says "receiving character data".

    But simply decrypting, or putting the data together (into one huge packet) doesn't seem to do the trick. Unless I missed something important.

    AntwortenLöschen
  2. The compress calls are actually well explainable: PNG. DDO links to it (and I am almost certain LOTRO does too) and it requires zlib1, so alas, both games call it everytime they load a PNG. Do you have IM or IRC, so we perhaps can talk about those very similar games?

    AntwortenLöschen
  3. From what I remember of the various packet dumps from LOTRO we had, characters names were transmitted in plaintext during login. That might have been changed though.

    I was wondering if it was possible that the client copied the RC4 decrypt function out of the crypto library so that it was impossible to hook calls to it after it is accessed the first time.

    When I remember, I'm on IRC: irc.efnet.net #lotroemu

    AntwortenLöschen
  4. It has changed. The login packets are now encrypted as well. But let's talk further in IRC.

    AntwortenLöschen
  5. Hello,

    i finally made it to decrypt the most of the server packets i collected during a session. Two bytes at the beginning of each packet are unclear to me, but i think i will figure this also out. At the moment i'm writing a little prog which automates this procedure. I will take a look if it could be used to encrypt own generated packets as well.

    Some things i can say clear:
    1. The CryptoApi isn't involved. (Maybe through a workaround? I searched google for similar methods but found nothing)
    2. Charactername, Servername eg. Maiar[DE], IP, Loginname are inside this packets. There are 2 urls inside.

    The URL's are as one string, the other vars are as Unicode formated(bigendian), with the length of the string given at first byte. The character values seem to have their own format, can't say anything yet, because i haven't tryed to (re)encrypt and feed the client.

    When this part is done i will post a link for a download location. 'Til a fully working server could be written there will be a lot of more things to figure out.

    The code is written in C# and will work only on win, but if i understand mono correct this could easy be ported, because i use no win gui or special methods.

    Greetings

    -T-

    AntwortenLöschen
  6. Congrats, that's great news! I wish I had more time to work on this at the moment :(

    From what I remember (this might be out of date now), the first two bytes of all the received packets are always the same, and are set up in the first received packet.

    The first two bytes of all sent packets are also always the same, and these are set up in the first received packet as well.

    AntwortenLöschen
  7. Sorry, took a little bit longer to put it all together.

    I got good news for you: you can also decrypt client packets as well now!

    The reason why it took so long is, that i tried to (re)encrypt the packets too. And i got it working. But... There seems to be some checksum, so that a packet with modded values can't be "injected". Or i've looked over something important. The bevore decrypted packets look like the encrypted after running through the algo.

    I tought long time if i should release the encryption source too, but i didn't this time. I fear that this could get into the wrong hands and people are trying to make cash with private servers. If you seriously need it and have good intentions then i will release it too.

    I also put an doc inside the zip of collected infos of some packets. Feel free to add something and re-release it so that we got one source for information. Btw. if you push the join world button, the client connects to an other server. This packet i tried to mod, so that i could connect afterwards to my server.

    If you got questions, i will answer it here.

    http://www.2shared.com/file/TKOhRKOk/LOTRO-Tools.html

    pw: bwgypyth.blogspot.com


    The easiest way to decrypt is to run the ".net packet logger" inside the zip while you play the game. Look at the source its not to hard to understand. The creators write that this lib also works under mono. When i will set up ubuntu again, i will try to port it.


    Greets

    -T-

    AntwortenLöschen
  8. sorry had to delete it, because i left some private data ;-)

    here again:

    http://www.2shared.com/file/PqsfbdiW/LOTRO-Tools.html

    same pw.

    md5: 92e242dd263cda3908b0f05b70ea130f
    sha-1: e9d6ba691495d180e430e8489af92e782f7a0874

    AntwortenLöschen
  9. This is a great piece of work.

    I remember the checksums gave xbadc0de a lot of problems. I don't think we ever solved how they were being generated.

    AntwortenLöschen
  10. Hi,

    solved the checksum problem. it was really dirty. also with lookup tables. it's depending on the packet number, the length of the "head", one static value and some summing. the next big problem: i fear that every entry in the datablock is also checksummed. somehow it looks like that. this is why they look so strange...!

    the easiest workaround, you only have to patch one byte in .exe and checksum protection is snow from yesterday. this saves a lot of work where you can make mistakes. you can inject everything (change chara names, change the join world server to yours ...)

    btw. encryption is working for 100%. there was a little mistake at my side, but now it's corrected and does like it should do. encryption routines will be "released" soon. if someone wants to make a ps this won't be a obstacle.

    summery what can be done til now:

    a.) get the correct checksum for whole data block in packet(the size is given in the beginning)
    b.) generate checksum for head (the first 20 bytes in packet, the checksum is 2 parted: 4 bytes vary, next 2 increase, last 2 bytes stay the whole time)
    c.) encrypt+decrypt client and server packets.

    Greeting

    -T-

    AntwortenLöschen
  11. Grats. Lookup tables were what xbadc0de used in the end.

    Does the exe patch need to be permanent, or we can step around it with an injected JMP after the client is started?

    AntwortenLöschen
  12. sorry my mistake i changed two bytes... look for 00 00 00 8B 4D 0C 3B C1 75 2D inside the newest version of lotro.exe. You should find it only once. Change 75 2D to 90 90 so that the check is removed and every packet header chksum is handled as valid. then you can change your character name without problems. don't forget to change the beginning length and data block length. i don't know where to find values like character level, hp, vip status, ...

    at the moment i haven't got much time. Last time i wanted to write a more generic server(with fake values), so that you don't have to log packets and use them to find things out. This piece of work must wait. sorry. But I can upload the encryption routine and the not final checksum validation for head checksum if you are interested.

    -T-

    AntwortenLöschen
  13. I'd be happy to get my hands on the encryption code. I currently don't have much time myself but if you upload the code (or send it to me) I'll put it into the gitorious repository.

    Keep up the great work!

    AntwortenLöschen
  14. Here it is:

    Class to (re)encrypt server and client packets again, plus a updated helper class for calculating the checksum of the data block of a packet. The head checksum generation(remember: mix of packet sequence nr look up in a table plus the 4 bytes at the right side of checksum) isn't working correctly til now. further invests are needed.

    http://www.2shared.com/file/7CVQl7GY/LOTROS2012.html

    -T-

    AntwortenLöschen