When tAmMo did his initial analysis of the packets he figured that headers during setup were 22 bytes long while the ones afterwards are only 20 bytes long. Until recently we went on with this scheme. When I was dumping the traffic between the server and my client I sometimes got very strange packets. These packets seemed to have the 22 bytes header! This did not make sense because only the unencrypted packets have 22 bytes in the header. Also these packets failed to make sense after decryption. Then it struck xbadc0de and myself: The server sometime decides zu send unencrypted packets!
So what does this mean: We don't have to figure out encryption. My dummy test server (which is just able to replay captured data) just sends out unencrypted packets and the clients just uses them. As easy as is. This saves us lots of time figuring out encryption, saves us compution time on our server and also means we don't harm the productive servers.
But how does the client detect an unencrypted packet coming in: Well, it's pretty easy. The third byte in the packet header is set to 0x00. It took me long hours to figure that out (and xbadc0de gave me the necessary hints, thanks!). I patched our decryption logic to deal with this fact.
https://gitorious.org/lotro/lotro-server/commit/52d3120caf0413e395549ba68f395cf88568ebe9
Donnerstag, 21. März 2013
Montag, 18. März 2013
Improving communication using IRC
Until now the complete conversation was done using comments on this blog. This has a severe impact on communication latency because answers to the simplest questions just take very long. Next to that answers to blog posts are often done anonymously which causes the commenter not to get informed by email about responses.
Today we started to improve that situation by using the IRC channel #lotroemu on the efnet servers. The idea was initially brought up by xbadc0de and him pointing out the massive problems in our communication style described above. We had some good discussion on the packets and I gained some great insight on how they are made up (and we also agreed on a common wording so we actually talk the same language).
From now on I'll try to join that channel as often as possible and I can only recommend this for everyone interested, too. Feel free to ask questions. Feel free to share opinions. Feel free to give advice.
See you on IRC.
Today we started to improve that situation by using the IRC channel #lotroemu on the efnet servers. The idea was initially brought up by xbadc0de and him pointing out the massive problems in our communication style described above. We had some good discussion on the packets and I gained some great insight on how they are made up (and we also agreed on a common wording so we actually talk the same language).
From now on I'll try to join that channel as often as possible and I can only recommend this for everyone interested, too. Feel free to ask questions. Feel free to share opinions. Feel free to give advice.
See you on IRC.
Sonntag, 17. März 2013
New server code uploaded
Today tAmMo sent me over his latest code which I uploaded to our git repository. The code should match the state of the binary used to capture the packets earlier, but fails to compile right now due to incompatibilities with the Helper.dll (even the one from the server does not work).
If you want to use the code the problems right now are:
Helper.HelperMethods.Instance does not exist.
Helper.BEBinaryWriter.WriteUInt16BEX does not exist.
Other than that the code import seems to have worked flawless :-)
If you want to use the code the problems right now are:
Helper.HelperMethods.Instance does not exist.
Helper.BEBinaryWriter.WriteUInt16BEX does not exist.
Other than that the code import seems to have worked flawless :-)
Analyzing the character creation
Now that we have a server working for logins it's time to start analyzing the packets. Since we currently only reach the character creation screen I took the time to analyze the packet sent by the client to the server when creating the client. This packet already has a wide range of variety so I started looking at creating a male, human burglar from Bree.
You can take a look at this analysis at the following page:
http://bwgypyth.appspot.com/packet.jsp?packet=305
I was already able to identfy some parts of the packet, e.g. hairtype, headtyp, mouthtype, etc. There are some interesting things to be seen in the different packets. First of all the packet differs even on same input data. Take you look at:
http://bwgypyth.appspot.com/packet.jsp?packet=9001
This is exactly the same action as performed in the other packet (creating a male, human burglar from Bree) but is padded with addition unknown data. So first of all we needed to identify the magic number in the packet that tells the server to create a character. Since we are assuming that we are facing an opcode based network protocol I was looking for a repeating sequence, named opcode.
For now we assume that opcodes are either 4 bytes long (or 2 bytes prefixed by the number of opcodes in the packet). 0x01 0x00 0x01 0x04 was the first data packet (after the 20 bytes of header code) in the the shortest captures. This opcode also appeared in all captures, so I assume this is the opcode to start a character creation.
This opcode (which appears to be the last opcode in the packet) is followed by 147 bytes of data for a male humanoid burglar character. Take a look at the data I gathered and please verify my assumptions. The best way to do this is by comparing the data (e.g. by using Meld which is an awesome diff viewer).
Another interesting catch is that female humanoid characters are created using shorter packets (only 128 Bytes). One of the reasons seems to be that the GUI lacks a method to add facial hair which seems also to be lacking in the packet (dwarf ladies might be different, I haven't checked that yet).
Have fun browsing the packets. I'm planning on investigating this packet further and keep you updated.
You can take a look at this analysis at the following page:
http://bwgypyth.appspot.com/packet.jsp?packet=305
I was already able to identfy some parts of the packet, e.g. hairtype, headtyp, mouthtype, etc. There are some interesting things to be seen in the different packets. First of all the packet differs even on same input data. Take you look at:
http://bwgypyth.appspot.com/packet.jsp?packet=9001
This is exactly the same action as performed in the other packet (creating a male, human burglar from Bree) but is padded with addition unknown data. So first of all we needed to identify the magic number in the packet that tells the server to create a character. Since we are assuming that we are facing an opcode based network protocol I was looking for a repeating sequence, named opcode.
For now we assume that opcodes are either 4 bytes long (or 2 bytes prefixed by the number of opcodes in the packet). 0x01 0x00 0x01 0x04 was the first data packet (after the 20 bytes of header code) in the the shortest captures. This opcode also appeared in all captures, so I assume this is the opcode to start a character creation.
This opcode (which appears to be the last opcode in the packet) is followed by 147 bytes of data for a male humanoid burglar character. Take a look at the data I gathered and please verify my assumptions. The best way to do this is by comparing the data (e.g. by using Meld which is an awesome diff viewer).
Another interesting catch is that female humanoid characters are created using shorter packets (only 128 Bytes). One of the reasons seems to be that the GUI lacks a method to add facial hair which seems also to be lacking in the packet (dwarf ladies might be different, I haven't checked that yet).
Have fun browsing the packets. I'm planning on investigating this packet further and keep you updated.
Samstag, 16. März 2013
Improving the packet analyzer
The packet web based packet analyzer gained one important feature today: interactivity. Until now it was just displaying a single packet. You couldn't do anything except look at it.
Today I finished converting this simple HTML-page into a full blown wep application. It is now possible to upload new packets and annotate them! The whole workflow is still rough around the edges but it's already a usable prototype. I uploaded the first 2 packages as found in the PDF and also corrected them for some mistakes (either in the PDF or made by me).
The model behind this application is pretty basic right now. The top-level entity is 'Packet' which represents a single network packet. Each packet can have one or more 'Analysis' attached, which group together 'AnalysisEntries'. An entry itself basically is a description of the bytes found in the packet.
Please test the application and upload your data to: http://bwgypyth.appspot.com/
The next step to be done is allowing more than one analysis per packet and to comment on packets, analyses and entries. And also editing existing entries is planned.
Next to that we now have a working server that is capable of dealing with starting the client, character selection and character creation! This really is awesome news and a major achievement of tAmMo. We'll upload the updated code as soon as possible. I'll let you know when this happens.
Today I finished converting this simple HTML-page into a full blown wep application. It is now possible to upload new packets and annotate them! The whole workflow is still rough around the edges but it's already a usable prototype. I uploaded the first 2 packages as found in the PDF and also corrected them for some mistakes (either in the PDF or made by me).
The model behind this application is pretty basic right now. The top-level entity is 'Packet' which represents a single network packet. Each packet can have one or more 'Analysis' attached, which group together 'AnalysisEntries'. An entry itself basically is a description of the bytes found in the packet.
Please test the application and upload your data to: http://bwgypyth.appspot.com/
The next step to be done is allowing more than one analysis per packet and to comment on packets, analyses and entries. And also editing existing entries is planned.
Next to that we now have a working server that is capable of dealing with starting the client, character selection and character creation! This really is awesome news and a major achievement of tAmMo. We'll upload the updated code as soon as possible. I'll let you know when this happens.
Samstag, 9. März 2013
Initial attempt at packet analyzer
Our biggest issue right now is to understand how the packets are assembled when they are tranfered from and to the server. Our previous attempt was to document the packets within an LibreOffice document hosted in our git repository. Sadly this does not work very well and especially does not scale. Only one developer can work on the document at a time.
Starting today we are working on a small web based packet analyzer. Right now it is capable of displaying a single annotated packet taken from our current documentation, which is a rather limited functionality. We are working on creating a server backend to store annotated packets so everyone can collaborate and either annotate packets or give feedback to annotated packets. You can take a very brief preview at the following URL:
https://gitorious.org/lotro/lotro-analyzer/blobs/raw/92df3298baa914602354df7501ea493e305851ca/packet.html
Note: The code is currently tested in Google Chrome, Firefox and Internet Explorer.
Starting today we are working on a small web based packet analyzer. Right now it is capable of displaying a single annotated packet taken from our current documentation, which is a rather limited functionality. We are working on creating a server backend to store annotated packets so everyone can collaborate and either annotate packets or give feedback to annotated packets. You can take a very brief preview at the following URL:
https://gitorious.org/lotro/lotro-analyzer/blobs/raw/92df3298baa914602354df7501ea493e305851ca/packet.html
Note: The code is currently tested in Google Chrome, Firefox and Internet Explorer.
Samstag, 2. März 2013
Updates investigation documents and server code published
I'm happy to announce that I just published the updated work of tAmMo containing updated documentation and servercode that should hopefully work with the updated network protocol from Riders of Rohan.
For now we are keeping the encryption/decryption checksum functionality proprierary. The primary reason for this that we don't want the official servers to get compromised. It is unlikely that one would be able to cheat using this code because we think that the official servers most likely have client packet validation beyond checksumming (e.g. the client sends a movement to the server and the server will validate that it is actually possible and reject it if not, as seen when you "bounce back" in the game). But since we cannot be certain about that we don't want to harm the official servers.
I'm in the process of creating proper documentation for how to run a client against your own server which will hopefully released soon.
For now you can find the code at: gitorious.org/lotro/lotro-server
For now we are keeping the encryption/decryption checksum functionality proprierary. The primary reason for this that we don't want the official servers to get compromised. It is unlikely that one would be able to cheat using this code because we think that the official servers most likely have client packet validation beyond checksumming (e.g. the client sends a movement to the server and the server will validate that it is actually possible and reject it if not, as seen when you "bounce back" in the game). But since we cannot be certain about that we don't want to harm the official servers.
I'm in the process of creating proper documentation for how to run a client against your own server which will hopefully released soon.
For now you can find the code at: gitorious.org/lotro/lotro-server
Abonnieren
Posts (Atom)