I see. So the color space itself is only 64 colors (I should've known, we're talking about the SMS silly me
). This definitely will make automated porting of colors from Lemmings 2 over to SMS a little tricky, especially with sets like Shadow where everything are just shades of dark blue
.
Anyway, on better news, I believe I've figured out trap modification, or enough of it for Pooty to start experiment supporting traps in custom levels.
The $DB97 value is actually an array index into a table that actually specifies the trap modification. This array starts at $3B1B (under 0-based indexing) with each element being 4 bytes. As you noted, 0 for $DB97 is treated special as "no trap modifications". A $DB97 value of for example 3 would therefore refer to the 4 bytes starting from $3B1B + 3*4 = $3B27. The $DB97 must be between 1-8 since right after the 8th entry of the array are more code, so there's only room for 8 entries.
The 4 bytes within an entry break down as follows:
byte 0: which type of trap. Must be 1-4 (all other values ignored by game programming):
1=squish (ie. Fun17)
2=noose (ie. Fun15)
3=beartrap (ie. Fun21)
4=the trap in the various Sega levels
byte 1,2: little-endian WORD specifying x-position (in pixels) of trap trigger location
byte 3: specifying y-position (in pixels) of trap trigger location
I haven't drilled deeply into the code the game does for detecting whether a lemming hits a trap trigger or not, but based on preliminary reading, I think the trap trigger is always 4x1 pixels, ranging from (trapx, trapy) to (trapx+3,trapy). It's easiest to see the relation by simply looking at the original values in the table, and see what location it corresponds to in the tilemap for the level. Below is the $3B1B table in the unmodified ROM:
$3B1B: 0, 0000, 0, (not used: 0 in $DB97 causes all the trap-related code to be skipped)
2, 02EC, 88, (Fun15/Taxing1)
1, 0160, 88, (Fun17/Taxing6)
3, 02A0, 80, (Fun21/Mayhem26)
4, 00CC, 80, (Fun20/Mayhem9)
4, 0224, 80, (Fun26/Mayhem12)
3, 02D0, 38, (Taxing2)
4, 0334, 70, (Tricky25/Mayhem13)
4, 0034, 60, (Tricky21/Mayhem18)
In case it isn't clear, the Sega programming's support for traps is extremely limited:
1) at most one triggered trap in the level.
2) it only has programming to explicitly support 4 different types of traps, namely the ones listed above.
So at this point (and likely well into the near future), please don't ask for changes to remove these limitations, as it will require major code surgery.