HowTo: Disabling checksum checks in the LOTRO client

To test a local server that does not support checksumming, you can patch you lotroclient.exe

Go to position: 3B 4C 24 1C 75 6A 8B

Replace "75 6A" with "90 90"

Moving the code from gitorious to gitlab

As some may have known: gitorious is shutting down. I decided to move our code over to gitlab. You can find it at:

https://gitlab.com/groups/lotro

State of the project

Dear readers,

I've continuously been asked whether or not this project is still alive. I'm trying to clarify the situation:

1.) I (bwgypyth) am no longer working on disassemling the protocol or creating a free server.

2.) xbadc0de succeeded in understanding the protocol (encryption and checksumming) and was able to create a minimal server. If anyone doubts this drop me an email (bwgypyth@gmx.net) and I will send you a screenshot to prove it (I'm not going to put the screenshot here). He visits from time to time #lotroemu on EFnet, but never shared any code with me. As far as I know he built the server in C++ on Windows and ported it over to Linux as well.

I am unwilling to redo the same work he did. If anyone is able to continue the project: He is. I can only hope he is going to share the code with us at any point in the future.

I'll still be monitoring this blog and will visit #lotroemu more often, but it's hard to do so working on two jobs 14 hours a day.

Finging strings in packets

As mentioned earlier the server will send some strings to the client to display. Today we'll take a look at how he does that and how.

First of all let's take a look how the strings are stored. I used lotrounpacker to take a look at the clients data. The site provides two binaries, the dat-unpacker and the localedata-extractor. The first one can be used to unpack client_local_English.dat, the later one will generate a large text file with all the strings. So how do we achieve that?

First of all copy DAT_UNPACKER.exe and datexport.dll to the root folder of LOTRO (it's a .NET 4 binary). After this simply call the binary from a commandline:

DAT_UNPACKER.exe client_local_English.dat

This command will run for some time (something like 10 mins), depending on your hardware and fragmentation of your hard disk. It will also eat about 600MB of disk. Once the command is finished you should have folder: data\client_local_English\2XXXXXXX\25XXXXXX\250XXXXX

Extract the LOCALDATAEXTRACTOR.exe into that directory and execute it (again a .NET 4 binary). It will generate the 30MB text file LocalData.txt containing all the strings. You can now browse this file in something capable dealing with larger text files (i.e. not notepad).

Now that we have the strings, how do we find out what the server is telling us? Basically the most simple way is to search for "25" in packtes. If we take the first 0x01000006 packet from the server and search for "25", we will find one occurence. If you now read 5 bytes backwards starting from the byte we found, we will read 250001FE0A. So we take a look at the LocaleData.txt to find "Authentifizierung wird überprüft ..." (in case of the client_local_DE, somehow the strings are wrong when using client_local_English). This is exactly the string the client is displaying during startup.

I also used a disassembler (ILSpy) on both binaries. The result does not provide us anything so far, but we might need to parse the files at a later point when the server is maturing. So far there still is a compile error due to an issue in the disassembling, but so far the code looks usable.

Analyzing the initial connection packets

I've spent the day analyzing the initial startup packets and how they are built up. First of all let's take a look at the order of the first seven packets:

The client starts by sending the server a 0x00010000 packet. This packet contains the clients version, the locale timestamp of the server in UTC and the login token obtained from the GLS server (this is the central server you are using for your login).

The server answers this by a 0x00040000 packet. This packets contains identifiers for the server, some seemingly random data, a test token to be sent back by the client, necessary instructions for checksumming and a constant.

Now the client will send a 0x00080000 packet to the server. The packet contains the test token from the server in reversed form and an unknown constant. From now on the connection is established and packets will be sent encrypted and properly checksummed.

After this step the server will start sending a large block of data, namely data worth 4 packets. The server will initiate this transfer with a 0x01000006 packet. This packet, again, constains the server identification and (at least in the case of my account) always the same data. The following two 0x00000006 (1, 2) packets are the same for my account no matter which server I chose. After this there always is another 0x00000006 packet, but it differes heavily depending on the server (or maybe other facts).

Further investigation is necessary, especially if the packets differ depending on the language of the client. It simply might be the case that Gwaihir has more information to tell than Vanyar, but this is unlikely. From what I remember we already know that these packets contain the strings (or better said the number of the string in a large list of strings known to the client) the client will display during the startup.

Initial analysis of the ping-pong packets

LOTRO sends "ping-pong" packets between the client and the server to verify the connection is established and both sides are working. These happen in the background without any user interaction. And these also happen in the menu. As far as I understood it so far, the client sends a "ping-packet" to the server and the server "pongs" it.
So far I've seen the following types of ping-pong-packets:

Fast-ping-pong (0x08004002)

This packet contains 10 bytes of data. The first 4 bytes is the sequence number of the last packet seen from the server (e.g. the last packet from the server was 0x0000001F, so this byte sequence will be the four bytes).
This is followed by 6 bytes which are currently unknown. The first four bytes always seem to be a number where only the first byte varies. Their purpose is completely unknown.
The last two bytes of these unknown bytes seem always to be increasing in small steps (between 1 and 4) so it might be the amount of seconds since the client started or even the sum of all bytes sent or something similar.
0x08004002 packets are (almost) always answered by the server with the same type of packet.

Full-ping-pong (0x0B004002)

This packet contains 22 bytes of data. The first 4 are againt the sequence number from the server. This is followed by a constant A§Úõ (at least during my capture) and a few other bytes. The byte at 0x1C always is the same value as the one at 0x28. The next 3 bytes are unknown, but the first always seems to be 0xA0. It might be that this might change once the last 2 bytes of the packet gets larger than 0xFF. Bytes 0x20 to 0x23 is an (increasing?) number that the server will have to put into his reply.
Again the packet ends with 6 bytes, which seem to be the same as in the 0x08004002 packet.
This packet is answered by a 0x0C0040002 packet.

Full-ping-pong-repsonse (0x0C004002)

This packet is the repsonse of the server to the clients 0x0B004002 packet. It constists of 18 bytes of data. The first 4 bytes are again the sequence number of the last packet the server saw from the client. The following 4 bytes are the copy of the bytes 0x20 to 0x23 from the clients packet. The following four bytes are unknown. Again the packet ends with 6 bytes, which seem to be the same as in the 0x08004002 and the 0x0B004002 packets.

Non-sequentiell-ping-pong's (0x0?000002)

These packets seem to be the same as their 0x0?004002 counterparts, except for the facts they are missing the 4 sequence number bytes at the beginning of the data. It is unclear why and when these are issued.

Clarification on the projects status

First of all thanks for the comments that show your interest! Unlike you might have thought I did not stop working on analyzing the packets sent between the client and the server. It's just that "real live" sometimes gets in the way ;-)

So I recently started capturing packets again to see how they are assembled. This is a time consuming process but I'll explain the very easy steps.

1.) Clone the repository from http://gitorious.org/lotro/lotro-server
2.) Import the project into your Visual Studio (Monodevelop is not supported right now)
3.) Run the Project LOTROPacketCaptureAndAutoDecryption
4.) Launch the game and "do something"
5.) Take a look at the packages generated by LOTROPacketCaptureAndAutoDecryption (you can also upload them to http://bwgypyth.appspot.com/ but be aware that they might contain sensitive data)
6.) Repeat steps 3.) to 5.) over and over again

We already know some of the details on the decrypted packets, like to be seen in http://bwgypyth.appspot.com/packet.jsp?packet=35004. All of the analyzed packets so far contain a header and data block. The header always is 20 bytes long (the initial packets sent have a header of 22 bytes, but more on that in a later post) and describes the data.

The data packets themselves can be distinguished by the "root command" (as we call it). This defines the type of data that is sent. So far we know the most about 0x00000006 packets, which are used when doing a character creation request.

The server and client also exchange "ping pong" packets to verify they can talk to each other and didn't loose the connection. I case someone is interested in understanding the process you can try to identify these packets (simply do nothing in the game and these will be the most frequent packages) and analyze how they are built up. Feel free to ask questions regarding this process in the comments and I'll try to answer them as quickly as possible.

And a quick sidenote: Everything we did so far is open source and can be found at http://gitorious.org/lotro. Feel free to play with the code and add things. The web application for visualizing packets is written in Java while the dumping of packets is written in C#. Both of these languages are pretty easy to learn (you don't have to deal with memory allocation), but feel free to contribute in whichever language you prefer.