Author Topic: Lemmings 2 Data Formats  (Read 3435 times)

0 Members and 1 Guest are viewing this topic.

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Lemmings 2 Data Formats
« on: December 01, 2021, 12:22:35 PM »
I'm currently working on something similar to the Lemmix player for Lemmings 2: The Tribes. (MS-DOS Version)

It's in the early stages so far, so still working on parsing the various file formats.

I have managed to fully parse and load the following data types; (With help from various notes/docs by geoo89 and GuyPerfect - Much appreciated)

- IFF Files
- RAW Bitmaps
- Styles
- Levels
- Non IFF Form files
- SBLAST.VOC
- All UI related files

I was wondering if anyone had done any work parsing .ANM files? (e.g. ARK.ANM)
Any info, would be greatly appreciated :)

Offline WillLem

  • Posts: 2423
  • "The truth is..."
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #1 on: December 02, 2021, 01:57:23 AM »
This sounds like a great idea! Will it be a simple Windows-ready hosting of the OG, or will you be adding QOL features (such as replay capabilities) to make it more accessible?

Apologies that I can't offer more than encouragement at this stage!

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #2 on: December 02, 2021, 09:15:59 AM »
The goal is to initially host the OG. However, all of the data is unpacked, converted then cached into something more friendly and modern to facilitate easy modding/addition of styles/levels/animations/sound and graphics data etc.

The player and tools will be windows initially and run on .NET Core. Aside from a thin wrapper around OpenGL to push the frame buffer, it's all managed. All rendering is done in a custom and fast blitter.

As for QOL elements, I am open to suggestions. I like the idea of storing replays :) (I'm already considering multiple fast-forward speeds for the impatient of us)

Now that I have all of the data caching working, I am underway with the player.

Progress: (A little rough around the edges)

- Menu/UI rendering
- Level rendering inc. objects with sub-components
- Rudimentary AI (They just get a random animation on spawn currently - red dot is collision)

« Last Edit: December 02, 2021, 09:26:40 AM by ravenix83 »

Offline namida

  • Administrator
  • Posts: 12071
    • View Profile
    • NeoLemmix Website
Re: Lemmings 2 Data Formats
« Reply #3 on: December 02, 2021, 08:53:48 PM »
Are you intending to replicate L2's physics exactly, or just to create a game that feels the same to the user (but might not exactly match how L2 works)?

Just because - if it's the latter case, I would suggest comparing your project to Lemmini rather than Lemmix, as one of Lemmix's most defining features is that it attempts to reproduce L1's physics exactly. (NeoLemmix and Lix would be even worse comparisons due to the additional features in the former, and open-source-overhaul nature of the latter.) That's not to knock it in any way - an L2-like game with QoL features and playable on modern platforms could definitely breathe some new life into the game - I'm just suggesting a wording that might give people a more accurate idea of what you're aiming for.
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #4 on: December 02, 2021, 10:01:23 PM »
It's a fair point. My intention from the start is to recreate the physics exactly (As far as possible). As to if I achieve that, is another matter entirely, given the number of skills to replicate. Currently using frame-by-frame analysis to determine physics.

The last thing I want to do is misrepresent or mislead.

How would you suggest I categorise it, if exact physics replication is a primary goal but not a certainty?


Offline geoo

  • Administrator
  • Posts: 1424
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #5 on: December 03, 2021, 08:29:24 PM »
Exciting to hear about this! I'm afraid I don't have any more information about the .ANM file format.
Is this needed for anything beyond the intros? If I recall, Lemmix started out as an editor with a playtest function, which I think is really what the community needed back then considering the focus has always been custom content creation, and the LemmixPlayers only came after (I might be wrong about this though).

Quote
It's a fair point. My intention from the start is to recreate the physics exactly (As far as possible). As to if I achieve that, is another matter entirely, given the number of skills to replicate. Currently using frame-by-frame analysis to determine physics.
I believe Lemmix was able to replicate the L1 physics exactly by using a disassembly and translating parts of it into Pascal, basically. I guess EricLang and ccexplore would be the people to know more about this. I don't know if anyone looked into disassembling L2, if anyone, I guess ccexplore would be the person to ask, though he hasn't been active in a while.

The main value of this is that people could use your L2 replica to do valid L2 challenges (e.g. save max amount of Lemmings, or use the minimum amount of skills to solve a level) with the benefit of the QoL features, as has happened with Lemmix in the past. At the same time, L2 has a whole bunch of weirdness that you might not want to replicate (and without a disasm probably wouldn't be able to replicate): For example, pausing can get lemmings stuck in cannons; you can pause in 1-frame intervals to bypass trigger areas of traps (e.g. to save 100% in Sports 1); the rules determining which lemming under the cursor is assigned a skill are dumb, etc...

So unless you want to replicate all of this, it might make sense to make some deliberate choices to get rid of some glitches along the way (e.g. use divers or hoppers to glitch into a wall).

As for QoL features: Replays are definitely a big thing for any community creating custom content. Multiple fast forwards sound useful too. Some more challenging features (that were definitely well appreciated when added to other engines) are savestates and going back 1 frame/second/some other interval. (Taking them into account already early during the design process might make them easier to implement in the future.)
Loading up individual levels via commandline parameters (and batch verifying validity of replays, if replays are implemented) should also prove useful for level designers. Might also be worthwhile checking in with kaimitai who was working on a new level editor/modding tool: https://www.lemmingsforums.net/index.php?topic=5769.0

Quote
The player and tools will be windows initially and run on .NET Core. Aside from a thin wrapper around OpenGL to push the frame buffer, it's all managed. All rendering is done in a custom and fast blitter.
Any chance this might reasonably run on Linux, e.g. using mono? Even within this forum, it feels like L2 is somewhat niche, and at least two people who appreciate L2 are using Linux :)

Either way, excited to hear about how this is progressing! Not sure I can be of much help, as I think you've already found all the documentation that is out there, but maybe I have some other stuff lying around or some obscure knowledge that occasionally could be useful...

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #6 on: December 03, 2021, 09:03:39 PM »
The .ANM files are for the intros and seem to be keyframe based. I've done about 30% of the work for it. But, as it's not critical, it's moved down the to-do list :)

I have been reading into and watching videos on known glitches. From my own experience there are quite a few undesirable quirks. I think you maybe right in that it would be better to make some deliberate choices about what is replicated.

Savestates are on the list. Not sure about rewinding, but it can go on the nice to have list :) (Savestates would help with that in the near term)

As for Linux support, it would work as its .Net Core 3.1 which works on most distros. I think the only snagging point would be the audio library. (I'd be happy to find a suitable cross-platform alternative, just need to dust off my Ubuntu VM)

Apologies, but the link you referenced is discussing L2 style limitations. Is this the right link?

I'm now at a point where all style, level objects and animations are working ( Just chains to go :) )

Offline geoo

  • Administrator
  • Posts: 1424
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #7 on: December 03, 2021, 09:25:42 PM »
Wow, you're making fast progress!

If you have questions about any specific glitches or behavior, feel free to ask!

As for Linux support, it would work as its .Net Core 3.1 which works on most distros. I think the only snagging point would be the audio library. (I'd be happy to find a suitable cross-platform alternative, just need to dust off my Ubuntu VM)
Nice, happy to hear that! ;)


Apologies, but the link you referenced is discussing L2 style limitations. Is this the right link?
Yes, in the topic kaimitai mentioned they were working on an editor/modding tool. I don't know if this still progressing or on ice, but I just thought it might make sense to make sure not to duplicate efforts or write incompatible tools.

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #8 on: December 04, 2021, 12:31:32 AM »
Thank you :)

Any info you have on known glitches would be fantastic. I have pulled pieces together, but don't like to miss stuff ;)

I have managed do an initial implementation for the chain simulation. Just making sure it's stable at any simulation speed.
« Last Edit: December 04, 2021, 12:44:38 AM by ravenix83 »

Offline namida

  • Administrator
  • Posts: 12071
    • View Profile
    • NeoLemmix Website
Re: Lemmings 2 Data Formats
« Reply #9 on: December 04, 2021, 06:38:15 PM »
Quote
I believe Lemmix was able to replicate the L1 physics exactly by using a disassembly and translating parts of it into Pascal, basically. I guess EricLang and ccexplore would be the people to know more about this. I don't know if anyone looked into disassembling L2, if anyone, I guess ccexplore would be the person to ask, though he hasn't been active in a while.

It was more like a clean room implementation - ccexplore used disassembly to figure out the exact physics, then described these to EricLang, who then implemented them with his own code.

This did initially lead to a couple of inconsistencies between DOS and Lemmix. Later patches by ccexplore and myself fixed some of these ("fixed" in the sense of "made Lemmix behave the same way as DOS") - most notably the nuke glitch but two other particularly notable ones are the pause-for-time glitch, and an inaccuracy in how long traps take to reset.
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #10 on: December 04, 2021, 10:28:14 PM »
In that case, seeing as I'm not familiar with DOS era assembly, I will be relying on frame analysis and using that to derive physics behaviour. So, to clarify on your original point. It is probably safe to say it won't be wise for me to frame it as being like Lemmix if exact accuracy derived from asm, was its defining feature.

Also, I will be looking to avoid the original L2 glitches.

For geoo - Removed all Windows dependencies. Now running on Linux (Surprisingly with higher framerates :thumbsup: - capping at 5k instead of 4.5k with vsync off)

Only dependencies are OpenGL 4.1 and OpenAL.

Offline Simon

  • Administrator
  • Posts: 3283
    • View Profile
    • Lix
Re: Lemmings 2 Data Formats
« Reply #11 on: December 05, 2021, 07:44:01 AM »
I'm the second Linux user with interest in L2.

Lovely project, I wish you the best as you push it forward. L2 tooling looks daunting to create, there are so many tiles, skills, ..., to support, and rewriting the L2 game is the holy grail.

Physics: I have no strong opinion on how close to DOS L2 is best. As soon as you change the behavior, it all becomes a question of taste anyway. Ultimately, one person has to decide what bug(fix) goes it and what doesn't. Let's see where this project takes you.

Quality of life: Replays and deterministic physics are good. Players like to share their replays with the level designers, so they can fix backroutes. Even during realtime play, consider to put all user actions into an internal replay first, and only then let the game display the outcome of that replay.

Savestates and replays are the ideal first quality-of-life features to make. Both together then lead to backwards framestepping: Lix and NeoLemmix implement backwards framestepping with frequent short-term savestates for performant short physics rollback (loading a savestate from ~1 second ago, then recompute forward to the desired frame using an internal replay), and more spaced-out long-term savestates to support occasional longer rollbacks.

Have you published the project source code? I'd be happy to compile and run it on Linux. Some years ago, I've compiled Nepster's NeoLemmix editor in C# with Mono. Thus, even if you haven't written any build documentation yet, I might get somewhere to give you feedback.

-- Simon
« Last Edit: December 05, 2021, 08:30:58 AM by Simon »

Offline namida

  • Administrator
  • Posts: 12071
    • View Profile
    • NeoLemmix Website
Re: Lemmings 2 Data Formats
« Reply #12 on: December 05, 2021, 05:50:49 PM »
Quote
Lix and NeoLemmix implement backwards framestepping with frequent short-term savestates for performant short physics rollback (loading a savestate from ~1 second ago, then recompute forward to the desired frame using an internal replay), and more spaced-out long-term savestates to support occasional longer rollbacks.

To be precise, in NeoLemmix's case, it creates a state every 10 seconds (as measured by the in-game clock, which in turn operates on the basis of 1 second = 17 frames), including at frame zero (because loading a state is much quicker than re-rendering / etc the level from scratch). Each time a state is loaded, all states newer than it are cleared. NeoLemmix will discard most states that are over one minute old, but will keep the ones that were saved on a xx:00 or xx:30 mark. Once states become three minutes old, they're only kept if they're on an xx:00 mark (these ones are never discarded due to age).

I believe Lix's algorithm is more complicated, primarily because whereas NL's was designed from the ground-up for singleplayer, I believe Lix's was originally designed for sync purposes in online multiplayer and then adapted to also help with single-player backstepping at a later date.
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #13 on: December 06, 2021, 11:43:09 AM »
@Simon - Thanks for the support. As for source code, it's currently not published. However, it is built as a self contained package including .NET runtime that will "just work" as long as OpenGL 4.1 and OpenAL is available on the system. Mono isn't used. Instead it is .NET Core 3.1. Testing how well this method works on other distros will be useful. Currently running on Ubuntu 20.04.

@namida - That sounds like a more sensible option. In terms of performance and reliability, I think 10 second snapshots are going to be the way to go as it needs to store the state of the world bitmap.

I have only just recently join this community and I'm finding it very welcoming and supportive :)

Offline namida

  • Administrator
  • Posts: 12071
    • View Profile
    • NeoLemmix Website
Re: Lemmings 2 Data Formats
« Reply #14 on: December 06, 2021, 04:56:45 PM »
Quote
as it needs to store the state of the world bitmap.

You need to store a lot more than this - basically the state of any in-game object, of the terrain map, and of attributes about the game itself (like remaining time, number of lemmings saved).

You're also going to have to think about how the fan will interact with save states - the fan very, very much blurs the line between UI and gameplay tbh. (Although, can you speed the fan up while paused? If so, perhaps saving the fan's state is not necessary.)

For reference - here's a list of the things saved in NeoLemmix. I've added the comments here myself (sometimes replacing the ones from the actual source code) to explain what they're for, if they're not obvious. There's also some redundancy in here - some is for performance reasons, a lot relates to outdated code and doesn't actually need to be in here anymore but just hasn't been tidied up yet.

Quote
      LemmingList: TLemmingList; // The lemmings and their current states / positions / etc
      SelectedSkill: TSkillPanelButton; // I believe no longer actually used for anything
      TerrainLayer: TBitmap32; // Visual copy of the terrain layout
      PhysicsMap: TBitmap32; // Physics copy of the terrain layout (ie: stores things like which pixels are steel, which are solid, which are one-way, etc)
      ZombieMap: TByteMap; // NeoLemmix has a type of lemming called a "zombie", this stores a map of which pixels are infectious (due to being too close to a zombie)
      CurrentIteration: Integer; // Or basically, "time elapsed since the start of the game, measured in frames rather than seconds"
      ClockFrame: Integer; // Number of frames since last whole second. Could really just be calculated by "CurrentIteration mod 17"
      ButtonsRemain: Integer; // Another NeoLemmix feature is locked exits which require pushing buttons to unlock; this stores the number that remain
      LemmingsToRelease: Integer; // Number of lemmings that exist on the level but have not spawned from an entrance yet
      LemmingsCloned: Integer; // Number of times the Cloner skill (which creates a clone of an existing lemming) has been used
      LemmingsOut: Integer;
      fSpawnedDead: Integer; // Kinda misleading name; it's how many lemmings were already zombies when they spawned (as opposed to becoming zombies later).
      LemmingsIn: Integer; // Number of lemmings saved
      LemmingsRemoved: Integer; // Number of lemmings removed from gameplay for any reason (death or saving)
      NextLemmingCountdown: Integer; // How many frames until the next lemming spawns from an entrance
      DelayEndFrames: Integer; // Gameplay will not terminate for this many frames, even if there are no lemmings left (used when the last lemming alive is killed by a triggered trap, so that the full animation plays)
      TimePlay: Integer; // Either "number of whole seconds remaining" (if level has a time limit), or "negative of the number of whole seconds taken" (if it does not).
      EntriesOpened: Boolean; // Whether or not the hatches have finished opening
      Gadgets: TGadgetList; // The various interactive objects (like exits, entrances, traps, etc) and their current states
      CurrSpawnInterval: Integer; // "Spawn interval" is Lemmings lingo for an inverse release rate (ie: low number = fast, high number = slow). NL can present either an RR or an SI to the user (it's an option), but generally uses SI internally.

      CurrSkillCount: array[TBasicLemmingAction] of Integer;
      UsedSkillCount: array[TBasicLemmingAction] of Integer;

      UserSetNuking: Boolean; // Has nuke been activated?
      ExploderAssignInProgress: Boolean; // Are the bomber countdowns from the nuke still in the process of being assigned?
      Index_LemmingToBeNuked: Integer; // Which lemming is the next one to start the nuke countdown for?

And then - lemmings in particular have about this many fields again per lemming. Objects, not quite so much, but still a few each.
« Last Edit: December 06, 2021, 05:05:44 PM by namida »
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #15 on: December 06, 2021, 05:31:28 PM »
Sorry if I wasn't being clear. I was meaning the world bitmap is the reason behind the interval as its the largest data element. I wasn't meaning
Quote
as it needs to store the state of the world bitmap.
in the literal sense of it being the only thing to save :). I appreciate the example of how others have done it so I can compare. The other addition I can think of is the state of the particle shower when lemmings explode in the Classic style.

As for the fan, I think storing the current fan speed would maintain consistency in expected behaviour when rolling back. Just so you don't lose as much altitude while waiting for the fan to spin back up again. Or it being abused by reverting while fan is at full speed.

So far, I have been making all of the gameplay objects serialisation aware, to help facilitate this.

Also, Zombie lemmings? Sounds like an interesting, but fun addition to Lemmings 1 :laugh:

Offline namida

  • Administrator
  • Posts: 12071
    • View Profile
    • NeoLemmix Website
Re: Lemmings 2 Data Formats
« Reply #16 on: December 06, 2021, 05:33:26 PM »
Quote
The other addition I can think of is the state of the particle shower when lemmings explode in the Classic style.

NL tracks this as a property of the individual lemming, so it's included there via LemmingList.

Quote
Also, Zombie lemmings? Sounds like an interesting, but fun addition to Lemmings 1 :laugh:

You should check out my "Doomsday Lemmings" level pack. ;) It's nothing but zombie levels.

The concept is simple enough - they act like normal lemmings except they don't exit (and don't trigger a few beneficial types of objects, such as the aforementioned exit-unlock buttons) and infect other lemmings they come in contact with. Also you can't give them skills - though if they were already performing a skill when they were turned, they'll still complete it, and they still retain their permanent skills.
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline WillLem

  • Posts: 2423
  • "The truth is..."
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #17 on: December 06, 2021, 09:05:18 PM »
Take it a step at a time. Get the game down first, then worry about QOL and UI stuff, that can come later once you have a much more in-depth idea of how it all fits together :thumbsup:

Offline namida

  • Administrator
  • Posts: 12071
    • View Profile
    • NeoLemmix Website
Re: Lemmings 2 Data Formats
« Reply #18 on: December 06, 2021, 11:05:16 PM »
Take it a step at a time. Get the game down first, then worry about QOL and UI stuff, that can come later once you have a much more in-depth idea of how it all fits together :thumbsup:

This is not necesserially the best plan. It can often be much easier to design around future features as you're going, than to try and fit them in later.

I guess you could think of an analogy with producing artwork - if you're drawing a lemming, that's going to be holding a machine gun, it's much easier to plan around the machine gun being there from the start and draw accordingly, than to just draw a lemming standing there then later edit it to be holding a machine gun. Not the best analogy but I can't think of a better one right now. :P
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #19 on: December 06, 2021, 11:57:33 PM »
An analogy, or an idea for a Lemming skill? :laugh:

The bulk of my time has been spent doing the groundwork for the unpacking/caching system. Now it's full steam ahead on the core game itself.

So far I have;

  • Level loading and selection (Selection screen currently cycles through all 120 levels showing the correct skills, title, time, style and level preview.
  • State serialisation (Basic framework)
  • Fixed update physics loop (Supports fast-forward and pause)
  • Basic UI support
  • Stage/bitmap supports layers (Background/Foreground), collision detection, removing terrain using brushes, marking terrain regions as read-only (steel), animated tiles and objects with hierarchy support.
  • Chain simulation
  • Basic Lemming entity with walk/fall/pop/bash abilities (Animations are all over the place atm until I finish compiling a list of what each one is)


Now I have the basic game loop, I'll be focusing on the Lemmings animations and skills. Logically, I am going to work through each level from 000, adding the required abilities to complete that level as I go.

Progress preview: https://youtu.be/Q20FKMVnS9c

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #20 on: December 07, 2021, 12:23:10 PM »
Does anyone have a Lemmings 2 MS-DOS save game with all the levels unlocked they are willing to share please?

@namida - Just been playing with NeoLemmix. Some fun levels in there. Although I had to enable online features for the Omega pack as it was complaining about missing styles. (Seemed to sort it). Was also having a dig through the bitbucket sources to see how you handled frame stepping and the skill projection overlay. (Very clean and readable code :thumbsup:) Pascal brings back some old memories. I moved away when they became embarcadero.

Offline geoo

  • Administrator
  • Posts: 1424
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #21 on: December 07, 2021, 02:10:36 PM »
Attached is a save game of mine; the first one (Test) has all levels unlocked with 60 lemmings.
Also attaching the savegame format description, in case it's useful.

btw, one other advantage of putting the source on e.g. github at some point is that you could get a PR for an L2 skill or two while you're still working on those :)

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #22 on: December 07, 2021, 02:59:35 PM »
Nice. Thanks for that.

I think maintaining L2 save game compatibility would be a good feature. I'll have a read through your spec.

As for sourcecode, it will be an option down the road once I have laid out the groundwork for sure :)

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #23 on: December 08, 2021, 01:58:37 PM »
@geoo - I have updated the l2save.txt with some of my own findings and to make it more complete. (See attachment)

Thanks for the file. I can now load/edit L2 save games :)

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #24 on: December 09, 2021, 10:27:13 AM »
I have almost finished documenting all of the Lemming animations and frame start/end for the segments within each animation.
From what I can tell, most animations are broken into several parts; (Majority just have the first 2 segments)

  • Left facing
  • Right facing
  • Floor aligned
  • Ceiling aligned
  • Sequence of frames relating to an Angle e.g. 0-90, 0-360 etc

I have come across 1 animation where for the life in me I cannot remember or find where it is used. Anyone able to shed some light on this please? (See .gif)

Many thanks

Offline Simon

  • Administrator
  • Posts: 3283
    • View Profile
    • Lix
Re: Lemmings 2 Data Formats
« Reply #25 on: December 09, 2021, 01:48:22 PM »
ArmWaving is the valve turner. :laugh: This appears only in DOS Sports 5, Blow Back...., when you interact with the steam valve in the top right corner of the level.

I thought it was half of a dancing animation, but that couldn't properly be, why would a dancing anim be cut in half.

-- Simon

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #26 on: December 09, 2021, 01:53:28 PM »
Makes sense now that you have said it :) ArmWaving was the best description I could think of at the time lol.

Thanks for the help.

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #27 on: December 15, 2021, 01:53:13 AM »
Just a little progress update. Main game loop is now in place and have begun implementing the first 8 skills required to complete Level000 (Classic 1).
Still a few quirks to iron out with the Miner (Clipped animation) and Builder (First brick offset) skills. The Lemmings can exit the level as triggers are fully working.

Here is a short video showing off skill usage and my progress so far;

https://youtu.be/4THvCC1z60Y

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #28 on: December 15, 2021, 11:38:57 AM »
For anyone interested, here are the test builds to go with the above preview;

They are built bundled with .NET runtimes. OpenAL is required to run.
To get started, copy your version of Lemmings 2 for MS-DOS into the data\dos subfolder that was extracted with the .zip. On first run you will be prompted to unpack and create a cache.

If you want music, you will need to copy .mod versions of the Lemmings 2 music into the data\music subfolder. (See Readme.txt within data\music)

To force a cache rebuild at any point. Start L2Player with the unpack argument.
Code: [Select]
L2Player unpack
Any feedback would be greatly appreciated :)

Windows Binaries: https://drive.google.com/file/d/1XX8m3ab7RY-jrHzdbx2ZZD4pqYs4EJAI/view?usp=sharing
Linux Binaries: https://drive.google.com/file/d/1FwohvDnKrJJGcR9aaUQ1pSHDngpStBMh/view?usp=sharing

OpenAL Installer for Windows: https://www.openal.org/downloads/
OpenAL for Linux (Ubuntu/Debian):
Code: [Select]
sudo apt-get install libopenal-dev

Offline geoo

  • Administrator
  • Posts: 1424
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #29 on: December 15, 2021, 05:26:14 PM »
This looks pretty nifty already! Needed to mess with the OpenGL version a bit ("export MESA_GL_VERSION_OVERRIDE=4.1COMPAT" did the trick), and it's running very smoothly for me :D

The most obvious thing I already noticed during the video is that the builder bricks are only 4 px wide rather than 6 px; maybe fixing this would also fix the issue with the gap? Related to the builder, L2 has a funny behavior where the brick of left-facing builders is placed differently from right-facing builders: For example, at a cliff edge, in one direction you can place the first brick so it protrudes 5 px, but in the other direction only 4. The collision detection of builders with terrain feels a bit too strict as well, but I can't say for sure.

A couple of other things you might already be aware of:
- You can't assign skills to lemmings that are currently performing a task.
- A few visual things: the animation of the floater is a bit funky. Also, permanent skills (floater, climber) invert the color of the lemming (green <-> blue) in the original game.

The Return key currently not mapped to fast-forward is very noticeable to me because I use it excessively in the original game, haha.

It's great how everything is extracted from the original game, no assets you need to include with the code, and any kind of modification to the game assets itself (I guess currently that's limited to levels) are automatically supported :)

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #30 on: December 15, 2021, 07:23:36 PM »
Thanks for the feedback :)

I have now switched to OpenGL 3.3 Core profile and removed the requirement for the COMPAT flag. (I was using Quad primitives instead of Triangle for drawing the Blitter Buffer texture to screen. Quad not supported in Core)

Good catch, my MakeBrick routine was placing the first 2 pixels on top of the first 2 which represent the brick in the animation. I'll take a look at the left/right differences too.

Currently the skill assign logic works as follows;

- Check if Walking state
- Check if already running a skill
- Check if already has a permanent skill
- Special case for Blocker

This is rudimentary and doesn't reflect the game as you know :)
At some point I will need to look into and compile a list of which skills can be interrupted. And which states allow for assignment of permanent skills. e.g. Kayaker - Must be in drown state

<Edit>: The pdf manual contains all of this info in a handy list ;)

The floater seems to be skipping the first few frames of the looping part. Should be a quick fix.

:D even when I played it on my Amiga 500 back in the day I never learned the keyboard shortcuts. Even on the DOS version. But I do appreciate its more efficient for Fast-Forward and Pause/Resume etc. I'll get those worked in for the next test release as I now have a PDF of the original game manual.

Yea, the idea is that the only asset dependencies are the original game files. Eventually the player will have a level and style editor so you can easily make new levels/objects/brushes and create new styles if you want to. However, any levels/styles created will be in the new player format. So will only work with the player. However, you can still import L2 levels made by other tools as long as they adhere to original game format/rules.

The next test release will let you test all 10 Classic levels and switch between them on the level select screen.

As always, I appreciate the time and effort people have put in to provide feedback and answer my endless and hopefully not too annoying queries :)

« Last Edit: December 15, 2021, 08:36:43 PM by ravenix83 »

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #31 on: December 18, 2021, 03:20:13 PM »
A quick preview from the filler class which will handle Fill, Glue and Sand...

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #32 on: December 18, 2021, 04:17:49 PM »
Test Build 2

The first 10 levels of Classic can now be played with all 8 skills available to Classic. The level select screen will let you continue past Classic 10 into the other Tribes. This is more for curiosity if you are interested in seeing the progress made to level and object rendering for the later tribes. You can choose the skills on the skill bar. They just won't do anything.

Windows: https://drive.google.com/file/d/11rOFAlbql-uzNOncFyQ6LBhw_iovVRxI/view?usp=sharing
Linux: https://drive.google.com/file/d/1Nt5dzERC6JPIS_JFu0e-GN8H76j49dRZ/view?usp=sharing

Note: If you have already tried the first Test Build, you would need to start clean with the cache directory and the root level directory with the binaries.

Changes

- Skill counts now updated when used.
- Digger/Basher/Miner can now change skills
- Press B to toggle Debug brush mode. (LMB paint terrain, RMB clear terrain)
- When building cache now reports which source DOS files are missing if any.
- Keyboard mappings added. (F1-F8, Space, P, Return, Z, X, Escape)
- Switched to OpenGL 3.3 Core (Removes COMPAT flag requirement on Linux)
- Builder collision checks fixed
- Builder brick offset bug fixed (2 pixels where lemming stood missing)
- Builder brick now overhangs 5 pixels facing right and 4 pixels facing left when stood on last pixel of a ledge
- Windows builds now include OpenAL
- Added level selection (Only Classic levels currently)
- Centered level title/time layout on level select screen
- Cleaned up cache format.
- Image data is now compressed.
- .NET runtimes are now compiled into the executable and extracted directly to RAM. (Much less clutter)
- Exploder skill now behaves accordingly when in Classic vs other tribes.
- Each tribe uses its own Exit animation.
- Floater animation fixed.
- Fast-Forward not resetting when changing level fixed.

Offline WillLem

  • Posts: 2423
  • "The truth is..."
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #33 on: December 21, 2021, 03:35:35 PM »
I'd like to test this but get an error message asking if I want to unpack stuff. I type "Y" for yes, and then nothing happens.

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #34 on: December 21, 2021, 07:09:35 PM »
Have you put a copy of Lemmings 2 for MS-DOS in the data/dos sub directory?

If you see my post for Test 1 it has further instructions. If that doesn’t work, please let me know.

Offline WillLem

  • Posts: 2423
  • "The truth is..."
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #35 on: December 23, 2021, 07:42:14 PM »
Have you put a copy of Lemmings 2 for MS-DOS in the data/dos sub directory?

If you see my post for Test 1 it has further instructions. If that doesn’t work, please let me know.

I will give this a try at some point, but I would very strongly recommend making installation of your clone a 1-step process with a clickable .exe that opens the game.

Until very recently, (Super)Lemmini suffered from a need to extract WinLemm resources for years, and its popularity has waned considerably over that time. Lemmings Revolution setups are also typically bogged down with resource/data faff as well.

I'd say "opens with 1 (double-)click" should be any developer's top, top, top priority when creating an app/program.

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #36 on: December 23, 2021, 09:02:21 PM »
I agree and I am open to suggestions. The reason it isn't a one step deal as I would have to bundle a copy of Lemmings 2 with it. An alternative would be if I make it so you drop a copy of the Player into your DOS Lemmings folder and it does the rest. Same as say Doom/Quake .exe replacements.

I'll update it for the next test build.

Offline Dullstar

  • Posts: 2032
    • View Profile
    • Leafwing Studios Website (EXTREMELY OUTDATED)
Re: Lemmings 2 Data Formats
« Reply #37 on: December 30, 2021, 07:01:39 AM »
Have you put a copy of Lemmings 2 for MS-DOS in the data/dos sub directory?

If you see my post for Test 1 it has further instructions. If that doesn’t work, please let me know.

I will give this a try at some point, but I would very strongly recommend making installation of your clone a 1-step process with a clickable .exe that opens the game.

Until very recently, (Super)Lemmini suffered from a need to extract WinLemm resources for years, and its popularity has waned considerably over that time. Lemmings Revolution setups are also typically bogged down with resource/data faff as well.

I'd say "opens with 1 (double-)click" should be any developer's top, top, top priority when creating an app/program.

It's not always feasible, and this is one such scenario, as creating such a program would require bundling L2's resources, which, while NL's continued existence shows it likely won't be an issue, is always a bit risky - while you might not get DMCA'd (among other possible penalties), bundling the assets certainly opens you up to it. That's why many of these sorts of projects have you provide the assets from an existing copy of the game.

Still, you can improve the process beyond simply telling users they need to drop things where you expect them. Instead, you can have your program check for those assets where you want them, and then if it doesn't find them there, you present a dialog that asks where Lemmings 2 is, and then have the program copy everything over to the expected directory instead of asking the user to do it. (Obligatory disclaimer: Not a lawyer. This is not legal advice. I'm simply mentioning what I've seen other projects do, so I assume this is the legally clean way to do it, but technically I don't actually know for a fact that this is 100% guaranteed not to get you in trouble).

Offline WillLem

  • Posts: 2423
  • "The truth is..."
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #38 on: December 30, 2021, 10:18:40 PM »
It's not always feasible, and this is one such scenario, as creating such a program would require bundling L2's resources, which, while NL's continued existence shows it likely won't be an issue, is always a bit risky

Sure, fair enough. By making the clone in the first place, though, you're already flirting with copyright. Might as well make it easy for people to access your clone by bundling the resources, making it fully functional from the getgo.

From what I've seen, DMCAs only tend to get involved when a project is either monetised, widely promoted, or otherwise attracting a lot of attention. Lemmings 2 is now abandonware and very unlikely to suddenly gain mass appeal, so the copyright owners have nothing to gain by pursuing legal action.

I am of course fully aware that these assumptions don't change the fact that the risk is there, I'm just saying it's a risk worth taking.

Offline Dullstar

  • Posts: 2032
    • View Profile
    • Leafwing Studios Website (EXTREMELY OUTDATED)
Re: Lemmings 2 Data Formats
« Reply #39 on: December 31, 2021, 05:05:53 AM »
Sure, fair enough. By making the clone in the first place, though, you're already flirting with copyright.

Not necessarily. I've seen this "you need the original game to extract the resources from" tactic used several times, with OpenRCT2 and OpenTTD being fairly high profile examples. Realistically, it mainly depends on who owns the rights whether or not you get DMCA'd for these types of fan projects: Sony *probably* won't. Hypothetically speaking, if Nintendo were to purchase the rights, on the other hand, of the currently supported clones, only Lix would likely survive the wave of cease and desists that their lawyers would fire off... well, except there's a bunch of disassemblies/decompilations of their games that also use the "original copy required to build" approach that they either haven't noticed, or haven't been able to do anything about. The entire concept of romhacks basically sustain themselves off of not actually distributing the edited ROMs; rather, they distribute a file that's basically a set of instructions on how to change the original ROM into the edited one.

It's not unusual for software not to run out of the box, though generally if it needs setup it should really provide an installer to take care of that (although tbh I suspect a lot of installers nowadays, at least for free software, are basically just for non-tech savvy users' convenience and don't actually seem to do much of anything except create a desktop icon and put all the actual filesystem clutter somewhere the user doesn't have to look at if they don't want to). OpenRCT2 will prompt the user to tell it where to look, and then it handles the process of copying everything where it wants on its own and you don't really have to worry about whatever it is that it's doing.*

*Actually, I don't remember if that's an OpenRCT2 feature or if it's part of the unofficial launcher a lot of people use for it, but either way, there exists a mostly automatic guided setup process for it.

Offline WillLem

  • Posts: 2423
  • "The truth is..."
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #40 on: January 01, 2022, 10:49:25 PM »
if Nintendo were to purchase the rights, on the other hand, of the currently supported clones, only Lix would likely survive the wave of cease and desists that their lawyers would fire off

Gosh yes, let's hope Nintendo doesn't have any interest whatsoever in Lemmings rights! :crylaugh:

It's not unusual for software not to run out of the box

I know, but I've always hated setup processes more complicated than "1. Install, 2. Run". So, if I have an opportunity to influence this aspect of a project (as in this particular case), then I'll speak up :)

Offline Ste Woz Ere

  • Posts: 54
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #41 on: January 02, 2022, 01:01:21 AM »
A Lemmings 2 clone? Yes please :thumbsup:

Even better if it (eventually) has player aids - while I managed fine using DOSBox with savestates, having NL-type stuff like rewinds, framestepping etc. would be even better. (might even make my L2 level pack a bit more palatable :P)
Tribes of Steel - a whole new Lemmings 2 game! (120-level pack)
Currently a WIP - all tribes are complete, testing and feedback is welcome!

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #42 on: January 04, 2022, 01:21:01 PM »
Happy new year all! Hope everyone had a good holiday all things considered.

@Dullstar / WillLem - I have reworked it so you can unzip the L2Player into your Lemmings 2 MS-DOS install folder and the rest is automatic. I feel this should hopefully be a good middle ground.
@Ste Woz Ere - Rewinds are on the roadmap along with a user friendly integrated level editor focused on ease of use and rapid iteration.
@exit - I had a go at working in your explosion physics code. Just in the process of comparing to DOSBox and need to fix a few quirks with reflecting y velocity. Also, with the explosion initialisation moving the lemming up 1 pixel with no regard as part of the setup, I have opted to fix this official glitch by having it do a collision check before moving.

Video link of the physics in action: https://youtu.be/PgAiXLyOeXw

Offline 607

  • Posts: 462
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #43 on: January 06, 2022, 09:51:07 AM »
Cool project! :thumbsup:

Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #44 on: January 06, 2022, 01:48:01 PM »
@exit - Great stuff! And thank you kindly. I'll take a look and let you know :)

@607 - Thanks, hoping to have tech preview 3 up within about a week or so.

Offline mobius

  • Posts: 2596
  • relax.
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #45 on: January 08, 2022, 12:22:14 AM »
first of all; awesome work :thumbsup: It's great to see somebody taking an interest in this after such a long time.

I'm not putting this in the "suggestions" thread because I haven't read through everything so I don't really know what you're plans are.
Just felt like saying that one of the major drawbacks to L2 imo has always been the very rigid game structure; that is; every level must have 60 lems and save them all (or as many as you can) etc. I'd be much more inclined to play the game if it was more like L1 in that reguard; giving more freedom to the level designer. Some puzzles only work with a smaller number of lemmings. And not have the number of lemmings in one level effect another level obviously.
everything by me: https://www.lemmingsforums.net/index.php?topic=5982.msg96035#msg96035

"Not knowing how near the truth is, we seek it far away."
-Hakuin Ekaku

"I have seen a heap of trouble in my life, and most of it has never come to pass" - Mark Twain


Offline RavenNine

  • Moderator
  • Posts: 49
    • View Profile
Re: Lemmings 2 Data Formats
« Reply #46 on: January 08, 2022, 01:27:20 AM »
Thanks. I noticed a lack of modern Lemmings 2 clones, so figured I'd fill the void :) Well, actually it was a flip between doing Lemmings 2 or Cannon Fodder. I wanted a retro game project get stuck into, and those two games were some of my favourites growing up.

Having the ability for custom levels to use varying Lemmings counts and remove the save them all per tribe to continue restriction is not a bad idea. Certainly for custom levels. Maybe have it so level packs can run in either Story mode, or Freeplay mode. Story using the 'save the tribe' format. And Freeplay allowing per level Lemming counts with no carry over between levels. Probably need some more thought, but I'm open to the idea.