[Bug][Editor] Editor changes level ID automatically

Started by IchoTolot, June 21, 2020, 10:54:18 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

IchoTolot

So, I've wanted to edir a United level with the level ID: xB93B6682B93B6682

After finishing and trying to check the replay, the player stated that the replay belongs to another level.

Back in the editor I noticed it had changed the ID of the level upon opening it. Edit: Pressing F12 seems to be the catalyst here.

I tried to insert the old ID into the editor from my replay again, but then the editor changes it to another completely new ID: 0000000100000002

I think it has to do something with the new ID text field and that it handles the lower case "x" incorrectly. Maybe in general it messes up lower case letters, the player acts normally though.

Going through my levels again, I noticed that the first letter of the ID is always an "x". "x3DFA23F13DFA23F1" for example. In fact every level I look at regardless of author has it. Maybe the "x" is something like a preamble and the editor handles it wrong. This would mean though that is bug is very critical as it messes up the replay connection between level and replays as the level is opened!

After more investigation: Including the "x", Some IDs are 17 characters long and some 16. Maybe the editor can only handle 16 characters and that's the cause of the breakage?
-- Tested it and indeed the editor leaves the "x" out when there are 17 characters or replaces it with a "0" when the level has only a 16 character ID. So the "x" is being handles wrong and the length also causes errors.



For now I will go around this bug by inserting the old ID in the levels textfile manually.

IchoTolot

#1
After being informed that the ID should always be exactly 16 hex long, I think I know what goes wrong here:

- The "x" is not displayed in the editor field.
- The 17 long IDs get displayed without the "x" in the editor and should be alright.
- The 16 long IDs get displayed with the "x" to make them 16 long in the editor and the "x" gets converted to a "0".

The solution would be to let the editor handle and accept both lengths. For new levels the correct 17 long IDs get used (x included) though, but it correctly reads and can handle the shorter ones.

There are many levels with this 1 character short IDs and I had no problem finding several in no time. So the editor at least recognising these should be supported!


But wait there is more as I noticed:

The level that broke had the correct 17 long ID and that confused me. I checked again and the result was stunning:

When pressing F12 the editor removed the first letter of the ID field! It was the "B" from "xB93B6682B93B6682".

So there is something very very fishy here. When I write in the "B" again and press F2 I get another ID similar to the "0000000100000002" one.

This also happens with "A". So I assume every letter at the first position gets eaten when pressing F12. ;)


Edit:

This could have something to do because some IDs violate the "first digit should be 0 to 7". But in another level "xB7B042E3B7B042E3" is transformed into "x37B042E3B7B042E3"

More examples (all without "x") from just the 1st rank of Clammings:

A9EB26C0A9EB26C0 --> 29EB26C0A9EB26C0
417EF0DFB6F0ED0 --->  0417EF0DFB6F0ED0  (case of shorter ID)
EF297435EF297435 --> 6F297435EF297435
C134398FC134398F --> 4134398FC134398F
B808D576B808D576 --> 3808D576B808D576
2F099531958BA69 ---> 02F099531958BA69 (case of shorter ID)
2795827B424E2E3 ---> 02795827B424E2E3 (case of shorter ID)
C8F6C9B7C8F6C9B7 --> 48F6C9B7C8F6C9B7
87DB31AF87DB31AF --> 07DB31AF87DB31AF (first number higher than 7)

Again, this is just the first rank of a pack with 30 levels in it!


The 3 error cases seem to be now:

- If the first number is higher than 7 it gets changed to a 0. This is activated with pressing F12 and the editor asks if the change should be changed after quitting.
- If the first sign is a letter it gets changed to a 0. This is activated with pressing F12 and the editor asks if the change should be changed after quitting.
- If the ID is one sign too short a 0 is placed in the first slot. This happens when opening the level and the editor won't notice it as a change.


So there seems to be many violations to the format namida once posted in discord that are not the user's fault! The editor simply assigned a somewhat faulty ID.

Namida's posted rules were:

- Should be unique in a pack.
- Exactly 16 hex digits long. (Often violated)
- First digit should be 0 to 7. (Often violated)
- At least one of the first 8 digits must be non-zero.

The error from the case that the first letter is being eaten is not covered in there.

The player is not buggy here, only the editor screws things up and changes your IDs. So the solution should be that the editor accepts and correctly displays more IDs that are outside of these rules (variable length, no first number limit, let's letters be at the first position). For new levels it can pick an ID according to the rules though.

namida

You're the techy kind. These rules were an attempt to explain it in a way that non-techy people can understand without needing to understand the underlying techy stuff. In actuality: The ID is an unsigned 64-bit value that must be between 0x0000000100000000 (2^32) and 0x7FFFFFFFFFFFFFFF (2^63 - 1). It's generally stored in hexidecimal format in the file, which is what the X prefix specifies. The reason for these limits are, on the lower end, to avoid confusion with older 32-bit IDs (ie: minimum is the smallest value that won't fit in a 32-bit integer), and on the high end, because some frameworks seem to have limited support for unsigned 64-bit values for whatever reason. (This might be less of an issue these days, to be fair.

Previous editor versions simply loaded and re-saved it as text. The recent versions now handle it as an actual number, so perhaps there's a bug somewhere in this. Prefixing with 0s is purely cosmetic.

It seems the issue here may have been the editor was previously assigning invalid IDs. No idea why it's changing values 0xA~~ and above differently from those 0x8~~~ or 0x9~~~.

I'll need to look into whether the game is actually using these invalid IDs as-is, or if it's invoking one of the "old ID format" or "no ID present" fallbacks, before determining the most appropriate fix.
My projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)
Non-Lemmings: Commander Keen: Galaxy Reimagined (a Commander Keen fangame)

namida

Okay so, it turns out NeoLemmix hasn't been enforcing this either, outside of during conversions from other formats. So - all that needs to be done here is lift these restrictions editor-side.

Done so in commit ee03680. I'll take a quick look at one other issue here, and release an update once that's handled or determined to be too complex for now.
My projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)
Non-Lemmings: Commander Keen: Galaxy Reimagined (a Commander Keen fangame)