LemSet - Custom graphic set compiler for DOS Lemmings (Sourcecode released)

[LJLPM]

As for loading it, simply allow people to change the set.

Sorry, but if I chose to disable the "tileset" list after having loaded a level, it's because of security against potential bugs (because tilesets don't share the same number of items!)  http://www.lemmingsforums.com/Smileys/lemmings/smiley.gif" alt=":)" title="Smiley" class="smiley" />
So the users who want to use the editor + custom tilesets for making DOS Lemmings's levels (.lvl files), will simply have to use already-existing tileset names for their custom tileset (ie: "special").

The YouTube video is very cool, by the way!  (it's nice to see fresh art for Lemmings) http://www.lemmingsforums.com/Smileys/lemmings/thumbsup.gif" alt=":thumbsup:" title="Thumbs Up" class="smiley" />

[namida]

Fair enough. I can see why it would be a hassle to code for that.

ccexplore: Awesome, thanks. At some point I'll work on recoding to avert that issue. Hm... would I be right in thinking then that the first code (in the stored order - the last code in processing order) would *have* to be a reference one in order for it to work properly, then? Of course, for VGAGR files you could simply add some junk data that's never actually referred to by any objects, but for other files (especially levels) that could be problematic (however, it seems that so far, everything's managed to deal with it).

Also - a thought on the unused bytes in the DAT files, since they're big-endian, isn't it possible that the data sizes are just 4-byte values?

[namida]

An interesting discovery about terrain. You know how I mentioned that colors that don't set the highest bit will display fine, as will those with a seperate mask? It does seem, however, that if the high bit isn't set for the color, that pixel will be regarded as non-solid by the DOS game (Lemmix does not exhibit this behaviour.)

EDIT: Also - I worked out what the second bit in animation_flags seems to be. If it's set, the object will animate at the start of the level, without being triggered. (If you set both bits on a trap, for example, you'll notice it animates once immediately when the level starts before sitting dormant and waiting for a lemming.) Lemmix crashes with this setup, so you'll have to try it in CustLemm or the original game.

Here's an updated version of LemSet that'll set the flags properly.



EDIT: Removed, http://www.lemmingsforums.com/index.php?topic=860.msg17749#msg17749" class="bbc_link" target="_blank">see here for an update which now also compresses the files instead of relying on an external tool.

[ccexplore]

Hm... would I be right in thinking then that the first code (in the stored order - the last code in processing order) would *have* to be a reference one in order for it to work properly, then?

I think the short direct encoding (ie. 00 nnn bytes ...) can actually also work there.  The game's decompression algorithm would've already read the byte that contains the "00 nnn" before it got overwritten, so even though the memory location containing "00 nnn" is already overwritten before processing the final 00nnn encoding, the byte containing those bits was already read earlier and so the algorithm can still pull the correct bits out from the read value (as opposed to having to read it from the memory location where it got overwritten).  Whereas the long direct encoding (ie. 111 nnnnnnnn bytes) will not work as a final processed code, because the "111 nnnnnnnn" cannot fit within one byte--the game would've already read the byte containing "111" but not yet "nnnnnnnn" when both become overwritten from processing the second-to-last code, and thus you lost the nnnnnnnn.

To be fair, it does seem like it could be theoretically possible to create LVL data that a non-aware compression algorithm would use the long direct encoding there.  (Indeed, even several consecutive short direct encodings there would become problematic.)  I will have to do some more testing and investigation to better understand this.

Of course, for VGAGR files you could simply add some junk data that's never actually referred to by any objects, but for other files (especially levels) that could be problematic (however, it seems that so far, everything's managed to deal with it).

The junk data approach is probably the simplest way to handle this for VGAGR files.  I don't believe a general method exists--in theory you could be given basically uncompressible data, such that there'd be no way to encode the data in the compressed format without taking up more bytes than the original data.  Whenever that's the case it'd be impossible for the algorithm to work correctly using the same memory buffer for both.

Also - a thought on the unused bytes in the DAT files, since they're big-endian, isn't it possible that the data sizes are just 4-byte values?

Well yes, the format may have been designed as such, but the DOS game definitely only uses the lower 2 bytes and ignore the upper 2.  It is probably due to the fact that real-mode DOS programs like Lemmings are 16-bit--conceivably the same file format could've been used across other platforms like Amiga, where 32-bit arithmetic might have been natively supported by the CPU and those ports could handle the data sizes as 32-bit values.

[ccexplore]

One more thing:  for VGAGRx data specifically, it looks like the maximum size of decoded section data that the game can correctly handle is 65535-16 = 65519, rather than 65535 as one might expect.

[namida]

Here's an update which now does the compression as part of the style building process. You no longer need to use an external tool. =D

EDIT: No need to post a new reply really. Another update; this one improves INI support and changes slightly how the pallete matching works (previously, if your custom pallete included a color that was also in the default pallete, it'd use the default pallete entry instead of the custom pallete one, which would result in the DOS games treating the pixel as non-solid even though it would display correctly). As for the INI - previously it had to adhere to a strict format (no blank lines, line feed at the end of every line including the final one, no comments); it should now be able to load INIs much more effectively. One final addition - you can now specify a number to use in the output filename (ground0o.dat and vgagr0.dat for example), previously it was hardcoded to use "groundXo.dat" and "vgagrX.dat". If no number is specified, it will still use those names as a default.


EDIT: I'm not sure if I've mentioned this anywhere, so next to the download seems the best place to put it.
When desgining terrain graphics, it should be noted that the first 8 colors (the default ones) will NOT register as solid in DOS games - they will display, but they will not act solid. Only the custom 8 colors will function fully. This does not apply to Lemmix; the entire pallete will function as solid there.

Source code can be found http://www.lemmingsforums.com/index.php?topic=860.msg18771#msg18771" class="bbc_link" target="_blank">here.

[ccexplore]

To be fair, it does seem like it could be theoretically possible to create LVL data that a non-aware compression algorithm would use the long direct encoding there.  (Indeed, even several consecutive short direct encodings there would become problematic.)  I will have to do some more testing and investigation to better understand this.

Here's an example.  The attached level will choke up on LemEdit (when loaded from a compressed source like a levelpak DAT file) and Lemmings.  In fact, it is possible to manually create this level from scratch in LemEdit, and save it into a levelpak with LemEdit, and still if you try to reload the level from the levelpak in LemEdit it will throw an error.

The level is created so that the initial portion of level data (which contains basically all the level stats like time limit, skillset, etc.) all uses unique values which also never appear anywhere in the rest of level data.  This forces the use of "direct encodings" as the last few encodings in processing order.  For simplicity I purposely kept the level itself is fairly empty (the only element is 1 entrance trapdoor) to help ensure that there is no way to encode the initial portion of level data by referencing bytes from those parts of the data.

Having understood this problem, I believe it is also possible to hack Lemmings to use separate buffers at least for handling levels, to avoid this problem.

[namida]

I've noted that BMPs saved with some programs (Paint.NET seems to often offend here) will automatically save as 256-color if possible, which LemSet is not compatible with. If you have this problem, open the image and re-save it in MS Paint, making sure to choose 24-bit.

[namida]

I haven't upgraded this or anything in quite a while, and same goes for the underlying Lemmings Functions libraries (I only ever really completed them to enough of an extent to make LemSet, to be honest).

Now - I will warn you, my coding is. MESSY. AS. HELL.
But if you want to brave it for whatever reason, I'm now releasing the source code for LemSet.
It's under my usual rules for anything I release open-source (or open-resource, as the case more often is). As long as credit is given, do whatever the hell you like with it.
Use QB64 to compile. I've only tried to compile it under Windows, but there are QB64 compilers for Mac, Linux and Android too, if anyone wants to have a go at porting it to those systems.


EDIT: Uploaded the source for dlgfxex too. It's also QB64. It doesn't use the Lemmings Functions library from LemSet or any other external files; everything is contained in the one source file. (It's also ten times messier than LemSet's code...)

[Mindless]

It's good for software to have version numbers http://www.lemmingsforums.com/Smileys/lemmings/wink.gif" alt=";)" title="Wink" class="smiley" />

[namida]

I only tend to use them on major projects. Stuff like Lemmings Plus, or my RPG Maker game... these kind of tools when made by me rarely tend to see more than a first release and then one or two bugfix updates. xD

[namida]

Here's an update! This adds support for making graphic sets with 32-color palletes. Of course, they will only work on engines that also support it, though with careful pallete selection it is possible to create a style that uses extra colors but also works on 16-color engines. It also allows pixel-perfect trigger areas.

I've left the previous version up, just in case anyone wants to be 100% sure of compatibility with 16-color engines (also, because its output is slightly smaller, in case that ever matters).

32-color palletes and pixel-perfect trigger areas require a compatible engine, of course. They will be supported by my updated Lemmix. I'll also release a CustLemmix update to support the 32-color palletes, but since the whole point of CustLemmix is retaining traditional mechanics, the trigger area update won't be applied. (The way I see it, purely-graphical enhancements do not break the traditional mechanics.) EDIT: Updated CustLemmix has been uploaded, check the CustLemmix topic.

[namida]

Here's another new tool, based mostly on the same code; a tool for editing MAIN.DAT.

http://www.mediafire.com/download/xac3o8fnqo72t3a/MainEditor.zip" class="bbc_link" target="_blank">http://www.mediafire.com/download/xac3o8fnqo72t3a/MainEditor.zip

Also, use this link for LemSet, rather than the attached files, as that one is slightly outdated:

http://www.mediafire.com/download/sa7nj936o0obfb9/lemset.zip" class="bbc_link" target="_blank">http://www.mediafire.com/download/sa7nj936o0obfb9/lemset.zip