Samstag, 28. Juli 2012

Revamping the work

I recently started hacking on lotro again, trying to figure out the checksumming. Right now I fail to understand how it works, eve hough tAmMo left some hints.

Dear tAmMo, if you read this, please get in contact with me. Thanks!

40 Kommentare:

  1. That's nice to hear, im thinking about joining again too ... maybe we should reactivate the #lotroemu channel on efnet again with a meeting that weekend.

    AntwortenLöschen
  2. Please keep this up! I have waited soooooo long for someone to get at least a sandbox up and running if nothing else.

    Please keep us updated.

    AntwortenLöschen
  3. We could focus first on redoing the gls part (as for me at least doesn't work with the client updated). I have already started this part, though I cannot get up the list of available worlds.

    AntwortenLöschen
  4. Got server list up (though servers appear down for the moment), and got authentication to show the error box when using wrong password :)

    AntwortenLöschen
  5. Ok, login stage finished, user & password verification, servers listed (with status up) and no issue entering the world queue for the selected server.
    Right now, it sends a request to the server (the loading screen before char screen) and stays there (for now)

    AntwortenLöschen
  6. Wow, thats a nice state of progress again. Do you use static packets or are you already able to calculate them on your own (packet de-/compression & chksum calc) ? I grabbed my testserver source this weekend and took a look at the packet/chksum generation again, did some cleanup and started to merge infos from now and then together. Lets meet on efnet #lotroemu some day ...

    AntwortenLöschen
  7. the login stage, user verification (to show the error box) and request to enter to the world queue is all handled by the asmx services and a pair of aspx pages

    AntwortenLöschen
  8. @xbadc0de, from what I can work out he's just generating the XML responses used by the launcher, nothing regarding the lotro client.

    I haven't worked on my server for a while, but I was successfully generating and encrypting packets up to the character selection.

    I stopped working on it before I had a chance to play with checksums, sadly.

    AntwortenLöschen
  9. I guess i will do some researches and updates to my old existing server code this weekend. Too bad that tAmMo seems not to be around anymore, so i have to do this checksum stuff all over again (encryption was already working on my old code), sounds like a waste of time, since he already found it out. Yeah well ... will be on #lotroemu if someone wants to join.

    AntwortenLöschen
  10. After 5 days & nights of continuous work on this stuff i have a complete rewritten server running.

    Running stable, multithreaded, with filtered 06 packets, movement partially working, no checksumming (using patched client).

    Gen and i where holding the #lotroemu chan, jcairn showed up for some time ... but in conclusion i have to say this blog is dead, sad but true :( And since we would need some more people to work together on this pretty coordinated i can only say what tAmMo said in July ... im out of this too, again. As long as there is no main group of people with enough enthusiasm and a public git and maybe a forum instead of a single user controlled blog i will keep my stuff on hold too.

    Its very sad that interest in this seemed to drop so much ...

    P.S.: tAmMo if you read this, please try to get in contact with me, i have some questions regarding the 0x00000006 packets, especially the 03 40 03 .. and 03 40 04 .. movement ones and the mechanism behind them (e.g. where is the height info in the client packets) ...

    AntwortenLöschen
  11. Whoah, don't think people aren't interested.

    AntwortenLöschen
  12. I might be a bystander with little or no know-how when it comes to coding, but I can seriously tell you that a WHOLE lot of people would be interested in creating at least a sandbox server for lotro.

    If you ever need a betatester, contact dakilven@gmail.com or deaforbust98@aim.com

    Keep on working, i'll check on efnet for you guys within the weeks ;)

    AntwortenLöschen
    Antworten
    1. I'm contacting jtcairn to see if I could help in any way. DO NOT abandon this project.

      Löschen
    2. I idle on IRC, if you send me a PM and you don't get a reply, I'm probably not signed in (I use a bouncer), and I won't see your message.

      Löschen
    3. Well, as you might have noticed, people dont do anything anymore. I updated my server code during the last weeks and archived it now at the following state:

      - working with latest Riders of Rohan client.
      - "heartbeat" packet receive/response mechanism working 100% (block count & size control).
      - movement working 100% (still no mesh collision detection, but heightmap usage at 100%). Normal walk, speeded walking, go-to-jumping to a new relative or absolute position, all working fine, even into unassigned areas. Character is being able to climb/slide down even the biggest mountains if wanted.
      - multithreaded multiplayer support (server accepts more than one connection, ppl see each other in the game is partly working)
      - complete toolchain for fetching, analyzing and converting packets. Server is not using sampled packets anymore, instead some sort of export from the packetanalyzer is used which is more flexible.
      - packet analysis done, providing bin-files for char selection, ingame UI config, chars representation within the world, etc. Includable in the above described export configuration.
      - Ingame input/output handling from/to all sort of messaging tabs/functions 100%.
      - Item handling partly working (add/inventory moving/destroy).
      - placement of mob/npc within the world partly working.
      - Alot of debug ingame control functions added ("dot-commands"), alot of debug output, either on server console or ingame if wanted. Mainly for packet inspection and to control server and character behavior.
      - Some sort of selfmade API to pass yet unknown packets from the client to any type of inspection mechanism thats needed. Currently i pass them to a perl script for deep packet inspection. Creation of response packets within that mechanism partly implemented.
      - All the implemented stuff that i forgot about right now :) ...

      Now im stuck. I tried to understand the char selection data, i can change name, level, race, class, etc, but not the layout or clothing for example. Weird format ... And of course the collision detection on the meshes would be one big project too.

      In conclusion, way too much work. It was a nice project, running fine for me as proof of concept and to chill around sometimes in the depths of middleearth (and the testareas of the developers :), but thats it. And since the IRC channel seems to be dead too after i left, well, dont need to say any more on this ...

      Unless people get their - erm - lower back - up and do something that will prove they're still interested in this project i will be out, since I was quite disappointed with the behavior of the people. I dont understand how they claim they have big interest in this project and tell me they're urged to work on that, but then i have to explain them the very basics of the packet types, which is of course the first mandatory step to do, and it seems they even didnt come up with the idea of doing that on their own or even try to understand the infos that are already available :/ ... Sorry if some people may be offended by that, but thats how i saw it.

      So, long story short, as said i archived this project now, moving onto the next one ...

      Thanks to the people on the (dead) IRC chan #lotroemu for their support, hints and testing.

      Over and out,
      xbadc0de

      Löschen
    4. Did you release this archive on github? I'd very much like to see this.

      Löschen
    5. I see you didn't. Would you be interested in releasing this work?

      Löschen
    6. Because, i'm not going to lie, it sounds like you have a playable release, a release i want to play.

      Löschen
    7. Is the source available somewhere?

      Löschen
    8. nah, i think it's private for himself. sad, happy new years everyone anyway. this project is officially dead i believe. (btw badc0de if you read this, please release it)

      Löschen
  13. Why should i? You wouldn't be able to do *anything* useful with it, this is no project for "bystander with little or no know-how when it comes to coding". This is no "ClickMeForFun.exe" with a fancy GUI, which gives you a 200 person private server where you can directly jump to the dungeon entry and kill draigoch or saruman in a raid like you may do it on the original, this is a bug-infested, private data driven, work-in-progress, never-finished piece of code at its very beginning, which would take another coder atleast two days to understand, and you ... well ... Once someone like tAmMo would show up (again), showing real interest in the project (and not just in the result) i would be happy to share the code ... the CODE, not some .EXE ...

    Sorry.

    AntwortenLöschen
    Antworten
    1. I'm interested in the code, not any binary. If you like to get your code up to the gitorious repository just send it to me on bwgypyth _at_ gmx _ net.

      I'm currently working on my own code but get distracted way to often :-/

      Löschen
    2. So whats your state of the code?

      Löschen
    3. "would be happy to share the code ... the CODE"

      Then share the code. Und suhle dich nicht in Selbstmitleid.

      Löschen
  14. My code currently does not much. It just ack's the packages it gets from the client, but I get a disconnect after a few seconds (at most a minute). I'm currently trying to understand the checksum-calculation, but I'm totally stuck :-/

    AntwortenLöschen
    Antworten
    1. You should not focus that much on the checksum calculation, its still very easy to patch the client to ignore it. There's muuuuch more other stuff to do and understand instead :).

      If you need the search/patch bytes for the latest client (like tAmMo provided for the old one ages ago) just tell me.

      Löschen
    2. I could definitely need some help patching the client. I'm currently using a very old version that I patched when tAmMo gave me the instructions. From what I remember I needed to patch 2 bytes to disable encryption checks and also replace the public key used by the client. But I can't remember how I did it.

      Any instructions would be highly appreciated :-)

      Löschen
    3. Hex-Search for 3b 4c 24 1c 75 6a 8b 45 and replace the 75 6a by 90 90. I never took care about the public key and never came across any problems with the key and communications. FYI: The (latest) client for Rohan has some major changes in the network-protocol, thus it wont work any longer with older server code (not even with old presampled packets).

      Löschen
    4. It's bad news that the network protocol changed. It might render our current information useless. I thought the protocol was changed when mounted combat was introduced, but I might be wrong.

      Thanks for the patching instructions. I'll look into this later today. But for now I'll keep my current client (Isengart standalone download, iirc) so I can at least continue testing my server code.

      From what I saw yesterday tAmMo also wrote code that generated jumptables from the raw packets. It might be possible that we could adapt to the changes by regenerating those tables.

      I also updated tAmMo's code with a couple of changes, see http://gitorious.org/lotro/lotro-tools if you like.

      Löschen
    5. The tables keept the same for enc and dec... The biggest change: Network Byte Order changed(eg. in the beginning, the server id was F1 00 and changes to 00 F1, same with the data payload length and all the other fields in payload header). Maybe this step was done because lotro could now be played on osx. You only have to reverse these (U)Int32,(U)Int16,... values. C# binaryreader uses little endian format, you'll only have to implement some "read the big endian format" methods.

      Checksum generation/validation for server/client packets keept the same, but it's too difficult to use and has no priority.

      My question:
      Has anyone figured out how to "enable" the admin interface or debug console inside the client (at character selection screen)? This could enormously help to find out which packet is for which action.

      Btw. the old asheron's call mmo game seems to use turbines protocol in a way. I looked at an old emu and found some similarities. Too bad that turbine never released the source in the past.

      Löschen
    6. Sadly I'm not aware of any method to do this and I'm not sure what the "admin interface or debug console" is. I guess you are referring to the chat window which allows you to use the "/whatever" commands?

      This seems to be only available if you are connected to the chat server and from what I know the connection to the game server is opened when you log into a character. But I don't know of hand.

      BTW: If you like to share your code with me feel free to do so. Otherwise I'll redo lot's of the work you already did.

      Löschen
  15. In the lotro.keymap there is an option "toggleAdminPanel" (CTRL+A) and also the DebugConsole, but there isn't shown anything on the screen when pressed. There is also a layout inside the client_general.dat file (...Chargen_AdminPanel_Button...). Ingame, maybe the chat console could be used, but there is also something for "pregame". Has anyone an idea how they have disabled it? My thoughts were a.) they disabeled the keys at character generation (CTRL+O is working at chargen) or b.) the client has to receive special packets which unlock this option. But then there must be some if then else function. I haven't found this function while debugging.
    (I am no professional in reverse engineering)

    The public key use is like xba said not important, only if you plan to authenticate users, who got an account on your p-server. This should be the last stage of such an server emu. There a other ways to implementing a auth mechanism without modifying the client.

    Code? There is a lot missing, but you can have it next days.

    AntwortenLöschen
  16. From what I know the logic in the client is implemented in lua. IIRC the ingame API provides you some features to develop plugins and the like. But this is only available when you are actually in game. I guess the AdminPanel and DebugConsole are only useable when in game, because the lua interface is only available then. But that's blind guessing.

    I'm aware that the public/private key is not yet necessary. It's just something I have on the TODO list to look into one day.

    Any code is good code :-)

    AntwortenLöschen
    Antworten
    1. Can anyone tell me whats the logic behind the packet generation?

      During session init, (udp)payload is seperated in 0x14 (0x16) 'payload header' and the rest into ONE 'payload data' segment.

      Now the difficult part: When you are in session e.g. a [00 00 00 06] packet (without burden), there could be more than one 'payload data' segment inside. For example the first server packet after session init, which contains the ip address. You got:

      [ 01 02 02 04 00 00 00 00 00 06 81 49 | 26 00 ...] where [26 00] is the length of following ip address data. At checksum calc, for every single data part a chksum has to be generated, this means for the server packet 4 checksums are calced and added. Which is the checksum for the whole payload data part.

      Now there are other payload data segemnt, which begin different:

      [01 10 01 | 00 04 | 05 03 04 00 00 00 00 00 06 | 59 00] (Client sends this after he received account/world database things) Again [00 59] is the length of the following data. [00 04] or only [04] means that there are 4 packets following. The data is to large for a single udp packet and split up.
      Next one is
      [ 01 10 01 00 00 05 03 04 00 00 00 00 00 06 80 EC 8|1 B0 | ]
      The only constant thing are the 00 00 00 00 00 06 bytes.
      Now at server side:
      [01 12 01 00 07 08 09 04 00 00 00 00 00 06 8|1 4A|]


      So, whats the logic behind that?!? Is this an extra header for each data part? Please comment.

      Löschen
  17. 00 00 00 06 is a root command specifying a set of subcommands in its data segment. And yes, each entry of this subcommands has a header- and datapart again, which, you guessed correctly, is taking part into the checksum. Headergeneration is somewhat strange, either on the root commands and the subparts, and some of the sucommands describe a multi-packet blob, with a current/max counter field in the subcommand header.

    AntwortenLöschen
  18. Mhmhm... difficult to write some good parsing/generating algo for this. Let's discuss some basic packets and correct me if i'm wrong:
    (H: means found in payload-header, D: found in payload-data, C: client sends, S: server sends)

    1. C/S H:[00 00 40 00] D:[00 00 00 10]

    Confirms the last received sequence no? But not every sequence no.

    2. C/S H:[08 00 00 02] D:[3C 00 00 00 DE FF]

    Confirms length (3C) of whole packet with (DE FF) - offset 0x10, 0x11 in reversed order

    3. C/S H: [08 00 40 02] D: [00 00 04 D4 | 2A 00 00 00 FA 0D]

    1 & 2 in one packet, first 1., then 2.

    4. C/S H: [0C 00 00 02] D: [00 1B 85 CC 00 00 01 C5 | 1E 00 00 00 16 0E]

    Second part confirms again length, but first 8 bytes?!?

    5. C/S H:[03 00 40 02] D: [00 00 00 10 | 41 A5 16 82 BC 78 5E 0F | 9F 43 A9 B2]

    First part sequence no, then 8 unknown bytes, then 4 unknown bytes. The 8 bytes maybe have something with time in commmon, but not in ticks? They start with 41 A5 ... and increase.

    6. C/S H: [0B 00 00 02] D: [41 A5 16 90 AF F8 DD CA | 00 1B 85 CC | 26 00 00 00 EE 0D]

    Last part is again length confirm, but the other two parts?

    7. C/S H: [0F 00 00 02] D: [41 A5 16 8D FC CC 45 17 | 00 16 3F F6 9F 59 A3 55 00 00 00 00 | 26 00 00 00 39 0B]

    Last part is again length confirm, but the other two parts?

    8. S H: [00 00 10 00] D: [00 00 00 02 00 00 00 1F 00 00 00 20]

    Request of server for lost packets

    0x02 - count of seq. no
    0x1f, 0x20 - seq. no of packets server missed and requests (again)

    9. C H: [00 00 00 07] D: [ ... ]

    Client resends a lost packet if there is 0x07 in header

    10. C H: [00 00 80 00] D: [ nothing ]
    Client sends "terminate session"

    Are there any other basic packets? How about a "server is full" packet?

    I Found this if you try to connect to server without auth before - [00 F1 00 08 | 00 20 00 02 | 00 00 00 05 DA 0E 88 2C FF E5 45 CE | 44 3B 58 00 00 00 00 00]

    I would be happy, if the "none coders" would fire up the packet logger / decryptor and try to start analyse protocol... make notes, find equal things and sum up everything they found out. On character generation they could figure out all the values, which change, when you create a chara with this hair and that body or race...
    This would help the project much more than requesting beta tests...

    AntwortenLöschen
  19. The root commands can be or'ed together, thats why 1. + 2. is the same as 3. I only take care about 3. and 10. for now, and print out some debug info on 8. .

    And yes, thats the main problem with nearly all the people i met here - pretend to be interested, but once you start talking techstyle or start explaining them what is necessary and how much work it will be, they're gone :-/. So far noone ever seemed to do even a little part of analyzing stuff, they took the sample code and packets, got them to work somehow and recognized that it would take them some work/brain power to bring this project further, so they dropped it. Real sad.

    AntwortenLöschen
    Antworten
    1. not everyone can code, most of us looked up lotroemu and got here and realized it was the only site still active.

      if you want help, i suggest you write out a meaningful guide on what the hell we should do step-by-step so easily a baby could do it.

      Löschen
  20. ooookeh ...

    http://ragecollection.com/img/ragefaces/surprised.png

    ... i think you didnt catch my point here.

    Never mind.

    AntwortenLöschen