Analyzing the character creation

Now that we have a server working for logins it's time to start analyzing the packets. Since we currently only reach the character creation screen I took the time to analyze the packet sent by the client to the server when creating the client. This packet already has a wide range of variety so I started looking at creating a male, human burglar from Bree.

You can take a look at this analysis at the following page:
http://bwgypyth.appspot.com/packet.jsp?packet=305

I was already able to identfy some parts of the packet, e.g. hairtype, headtyp, mouthtype, etc. There are some interesting things to be seen in the different packets. First of all the packet differs even on same input data. Take you look at:

http://bwgypyth.appspot.com/packet.jsp?packet=9001

This is exactly the same action as performed in the other packet (creating a male, human burglar from Bree) but is padded with addition unknown data. So first of all we needed to identify the magic number in the packet that tells the server to create a character. Since we are assuming that we are facing an opcode based network protocol I was looking for a repeating sequence, named opcode.
For now we assume that opcodes are either 4 bytes long (or 2 bytes prefixed by the number of opcodes in the packet). 0x01 0x00 0x01 0x04 was the first data packet (after the 20 bytes of header code) in the the shortest captures. This opcode also appeared in all captures, so I assume this is the opcode to start a character creation.
This opcode (which appears to be the last opcode in the packet) is followed by 147 bytes of data for a male humanoid burglar character. Take a look at the data I gathered and please verify my assumptions. The best way to do this is by comparing the data (e.g. by using Meld which is an awesome diff viewer).

Another interesting catch is that female humanoid characters are created using shorter packets (only 128 Bytes). One of the reasons seems to be that the GUI lacks a method to add facial hair which seems also to be lacking in the packet (dwarf ladies might be different, I haven't checked that yet).

Have fun browsing the packets. I'm planning on investigating this packet further and keep you updated.

Improving the packet analyzer

The packet web based packet analyzer gained one important feature today: interactivity. Until now it was just displaying a single packet. You couldn't do anything except look at it.

Today I finished converting this simple HTML-page into a full blown wep application. It is now possible to upload new packets and annotate them! The whole workflow is still rough around the edges but it's already a usable prototype. I uploaded the first 2 packages as found in the PDF and also corrected them for some mistakes (either in the PDF or made by me).

The model behind this application is pretty basic right now. The top-level entity is 'Packet' which represents a single network packet. Each packet can have one or more 'Analysis' attached, which group together 'AnalysisEntries'. An entry itself basically is a description of the bytes found in the packet.

Please test the application and upload your data to: http://bwgypyth.appspot.com/

The next step to be done is allowing more than one analysis per packet and to comment on packets, analyses and entries. And also editing existing entries is planned.

Next to that we now have a working server that is capable of dealing with starting the client, character selection and character creation! This really is awesome news and a major achievement of tAmMo. We'll upload the updated code as soon as possible. I'll let you know when this happens.

Initial attempt at packet analyzer

Our biggest issue right now is to understand how the packets are assembled when they are tranfered from and to the server. Our previous attempt was to document the packets within an LibreOffice document hosted in our git repository. Sadly this does not work very well and especially does not scale. Only one developer can work on the document at a time.

Starting today we are working on a small web based packet analyzer. Right now it is capable of displaying a single annotated packet taken from our current documentation, which is a rather limited functionality. We are working on creating a server backend to store annotated packets so everyone can collaborate and either annotate packets or give feedback to annotated packets. You can take a very brief preview at  the following URL:

https://gitorious.org/lotro/lotro-analyzer/blobs/raw/92df3298baa914602354df7501ea493e305851ca/packet.html

Note: The code is currently tested in Google Chrome, Firefox and Internet Explorer.

Updates investigation documents and server code published

I'm happy to announce that I just published the updated work of tAmMo containing updated documentation and servercode that should hopefully work with the updated network protocol from Riders of Rohan.

For now we are keeping the encryption/decryption checksum functionality proprierary. The primary reason for this that we don't want the official servers to get compromised. It is unlikely that one would be able to cheat using this code because we think that the official servers most likely have client packet validation beyond checksumming (e.g. the client sends a movement to the server and the server will validate that it is actually possible and reject it if not, as seen when you "bounce back" in the game). But since we cannot be certain about that we don't want to harm the official servers.

I'm in the process of creating proper documentation for how to run a client against your own server which will hopefully released soon.

For now you can find the code at: gitorious.org/lotro/lotro-server

Revamping the work

I recently started hacking on lotro again, trying to figure out the checksumming. Right now I fail to understand how it works, eve hough tAmMo left some hints.

Dear tAmMo, if you read this, please get in contact with me. Thanks!

Decryption and encryption progress

Recently a commenter name tAmMo started playing around with how encryption and decryption of the packets might work. He made several comments about his progress and just yesterday he posted something unexpected: Source code related to client packet decryption.

So what does this mean? Not much for the "end user", but a great deal for researchers. Also tAmMo did not release the complete code, especially the part of encrypting packets to be sent to the server. This could be used for hacks or to create a private server which is not the intention of this attempt.

The code is hosted at https://gitorious.org/lotro/lotro-tools for now. Take a look at it, play with it, and keep us updated using comments!

Regarding compressed communication

This blog has been inactive for quite a while, but users where still commenting. I didn't have the time to respond, but I kept watching the progress. Now is the time to sum up some of the comments.

Recently the idea came up if the communication between the client and the server might be compressed (additionally to being encrypted). I don't think this is the case. From my understanding the game does most likely use opcodes for data exchange. The amount of data is not very high so having the cpu overhead of doing stream encrypting does not really bring much benefit.

Our commenters have also tested to use a "verbose" version of the zlib that writes the protocol of the called method. From what we could observe only calls to "decompress" were made. Therefore it's most likely that the communication is only RC4 encrypted data.

I also was contacted by an Austrian developer that is looking into doing the same attempt we are doing for DDO. He is using easyhook (http://easyhook.codeplex.com/) to trace the calls. He focused on CryptImportKey() and CryptGenKey() to receive the private/public keys and the RC4 stream cipher. This might be an option for Windows users to figure out what the client is doing (Linux users can always use wine and hack it in whatever way they prefer).

I'm looking forward to what else our readers might come up with. Feel free to comment on this blog for whatever reason you like, e.g. connecting to other developers.