Author Topic: Lemmix  (Read 9828 times)

0 Members and 1 Guest are viewing this topic.

Offline EricLang

  • Posts: 464
    • View Profile
Lemmix
« on: June 23, 2011, 08:15:02 AM »
I'm currently writing a new Lemmix program (player and editor).
(For those who don't know: look at http://ericenzwaan.nl/eric/lemmings/lemmix.htm)
It will have it's own file formats and abandon the old dos files.
There will be a conversion tool for known versions (all dos versions, lemmini, winlemmings, and I hope for tribes lemmings)
It will be possible to extend the game by writing addons.
However to make the new program as complete as possible it would be nice to inculde all available lemmingstates for all known lemming games.

Of course I have these ones:
Walking, Jumping, Digging, Climbing, Drowning, Hoisting, Building, Bashing, Mining, Falling, Floating, Splatting, Exiting, Vaporizing
Blocking, Shrugging, Ohnoing, Exploding.

Is there someone who has or can make a complete overview of other states (and skills like climber and floater) in other games like in the Tribes game?

Offline Clam

  • Posts: 2187
  • Smiley: :8():
    • View Profile
Re: Lemmix
« Reply #1 on: June 23, 2011, 09:56:39 AM »
Hey, Eric! Good to see you're still around. 8)

The Lemmings Encyclopedia has a list of skills and game elements in Lemmings 2, which you should be able to work through and find all the possible lemming states.

Offline bombsite

  • Posts: 54
    • View Profile
    • JS Lemmings
Re: Lemmix
« Reply #2 on: June 23, 2011, 11:01:12 AM »
Hi Eric, I've been using Lemmix recently whilst trying to create my own Lemmings clone in JS and it's been a great help  :thumbsup:

I presume as Lemmix is still available and you're still continuing to develop it that you've never had any legal problems like some other Lemmings clones?

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #3 on: June 23, 2011, 11:07:29 AM »
Never had a problem with legal stuff.
Thanks Clam Spammer. I will list these actions and skills from that site.
Lemmix clone and editor is still available from the mentioned site yes.

Offline ccexplore

  • Posts: 5311
    • View Profile
Re: Lemmix
« Reply #4 on: June 23, 2011, 11:50:02 AM »
Is there someone who has or can make a complete overview of other states (and skills like climber and floater) in other games like in the Tribes game?

I doubt you'll find a comprehensive such list for a while, but you're welcome to try to make one yourself using information on the web.  If you download the game from abandonia, remember that you can use the practice levels to try out all skills (except blocker, but you already know what that does).  Keep in mind that a single skill can encompass multiple states, just like how for example a climbing lemming transitions to "hoisting" when it reaches the top.

Oh, and don't expect me to disassemble the game programming for Lemmings 2 any time soon if ever.  Sorry, but 50+ skills is just a bit much compared with 8, plus the skills in Lemmings 2 are clearly more sophisticated in general, and therefore takes more work to disassemble and understand.  You'll have to go the old-fashion way for now of emulating via obsevation and trial+error.

Incidentally, I think most people can agree that there are too many frivilous skills in Lemmings 2, so I don't think you necessarily need to go for implementing every one of them, unless you're dead set on recreating every single Lemmings 2 level.

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #5 on: June 23, 2011, 12:14:26 PM »
I have played Lemmings 2 long long time ago and do not remember much of it.
A special puzzle - programatically - is the relation between "skill" and "state".
I don't expect a complete disassemble of course...
All Lemmings 2 levels are on my wishlist though. Future thingy...
Currently I'm just setting up the system, so details will come later.

Offline finlay

  • Posts: 543
    • View Profile
Re: Lemmix
« Reply #6 on: June 23, 2011, 07:09:50 PM »
To be honest, and mainly for selfish reasons, I really think you should stick with the DOS formats, at least for custom levels in regular original Lemmings styles. It's just that this way keeps compatibility with Custlemm and stuff. I'd hate to see the day when only levels in the new Lemmix format are produced and I'm left hanging because my computer is a mac and can't run Lemmix or Lemmini...

Offline ccexplore

  • Posts: 5311
    • View Profile
Re: Lemmix
« Reply #7 on: June 23, 2011, 09:14:40 PM »
I'd hate to see the day when only levels in the new Lemmix format are produced and I'm left hanging because my computer is a mac and can't run Lemmix or Lemmini...

Hmm, I've always thought Lemmini, as it is written in Java, should be platform-independent, but it's quite possible I remembered wrong since I haven't really played it much.

Perhaps Eric could make his conversion tools such that it's possible to convert Lemmix-format levels back to DOS formats if the level only contains elements and styles available to DOS?

Anyway, it probably won't ever be the case that "only levels in the new Lemmix format are produced".  Consider Cheapo for example which was popular a few years (decade?) back, there were certainly lots of Cheapo levels produced that were not available in any form for CustLemm, yet that hasn't completely killed the CustLemm format either.  If anything CustLemm has outlived Cheapo. :-\

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #8 on: June 23, 2011, 09:45:33 PM »
Conversion of dos-like levels back to dosformats will probably be possible.
Another option could be to try and compile the program with FreePascal (Delphi clone), which is crossplatform.

The range of game-possibilities is much wider, when I leave the old restrictions.

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #9 on: June 23, 2011, 10:19:37 PM »
For example leaving the "graphic set" restriction will enable to mix tiles and objects of different styles.
Tiles and objects will still be named something like brick_001, crystal_005 etc.
This adds something to the game and removes nothing.

Furthermore the level can be saved in two ways:
1) containing only metadata: all needed graphics are stored somewhere else.
2) containing ALL information to play a game (tiles, objects, texts, lemminganimations, cursors etc.). When compressed the size will be (educated guess) 50-100 KB.

Offline EricLang

  • Posts: 464
    • View Profile
Searching for testers in the near future
« Reply #10 on: July 04, 2011, 10:59:36 PM »
First thing I am going to make is my new editor.
If real-life is not interveining in the near future, a first test version will be ready in a few weeks.
The first part will be creating (meta)data like
-terrains
-objects
-soundeffects
-music
-lemmings
If someone is interested in testing in this early phase and give some feedback, I would be very happy.
I could put him/her write down his emailadress and notify him/her where to dowload.
Send me a PM if you are a candidate tester.

Offline Simon

  • Administrator
  • Posts: 3879
    • View Profile
    • Lix
Re: Lemmix
« Reply #11 on: July 05, 2011, 12:51:05 AM »
Wouldn't it be best to make it open source and/or offer public binaries, to maximize early feedback?

Not sure whether I want to commit to provide regular feedback, but I'd sure like to follow the development process and give the occasional feedback.

-- Simon

Offline ccexplore

  • Posts: 5311
    • View Profile
Re: Lemmix
« Reply #12 on: July 05, 2011, 02:20:58 AM »
Wouldn't it be best to make it open source and/or offer public binaries, to maximize early feedback?

I'd say it depends on what level of quality and completeness is expected from these "early" versions.  I do agree there's great value for feedback especially on user interface design early on, so that time isn't wasted coding and testing stuff that's fundamentally flawed on a design level.  On the other hand, if the expectation of these early versions is that there could be a number of prominent bugs, a "too-open" early release can have the effect of not getting sufficient breadth of testing (ie. people stop trying the editor after running into bugs one too many times), or discourage more testing further down the line when things start to stablize (ie. people got impression from early release that it's buggy, not worth their time, and stop looking at it later on even when things got much better).  Similarly, releasing versions with known major incomplete areas "X" to general public may only serve to encourage feedback of the type "it doesn't do X", "X doesn't work" that aren't terribly helpful (again, the same can theoretically happen in limited release as well, but at least you get less of it).  There is the fact that not all user feedback is created equal, although that issue isn't really solved by limited releases either.

As for open source, in terms of maximizing early feedback I'm not sure how useful it is.  First consider the fact that it is only of interest to anyone who knows programming, and is familiar with the language being used.  Off top of my head, I'd say it is of use primarily only if Eric is committed to a Linux version and that he is not planning to release actual binaries for Linux version during early testing phase, in which case having compilable source is the only alternative left for early testing on that platform.  Even then it is not clear whether source release needs to be fully open to public, instead of only to the few committed specifically to that line of testing.

Ultimately I trust Eric has taken all that into consideration, and will figure out when he feels is a right time for general public release vs limited release etc. for testing.  I do agree with Simon that the earlier a public alpha release happens, the better, so long as there's a reasonable level of quality and completeness that makes it worthwhile for the general public to try out.

Offline Ron_Stard

  • Posts: 312
    • View Profile
Re: Lemmix
« Reply #13 on: July 05, 2011, 11:01:32 AM »
Will this new version have a port for Linux?  :D

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #14 on: July 05, 2011, 12:48:30 PM »
Lemmix will become opensource.
But more important: Lemmix can be extended by writing addons, for anyone who is able to write delphi code.

I have never ported my delphi code to another platform, so if I will do that it will be one of the last things i do.

The first feedback I need is mostly on userinterface stuff.
The early testversions will not be opensource, for abvious reasons.



Offline EricLang

  • Posts: 464
    • View Profile
Leming States
« Reply #15 on: July 15, 2011, 05:08:46 PM »
I gathered the following lemmingstates for Lemmix. The states 0..127 will be reserved. The states 128..255 will be "free" for addons.

These are the names which will appear in the editor, when editing a "LemmingDefinition".
If anyone has remarks on these names or finds omissions I would be glad to hear.

Code: [Select]
  DefaultLemmingStateMap: array[lsNone..lsTwister] of TIdentMapEntry =
  (
    (Value: lsNone;            Name: 'None'),
    (Value: lsWalker;          Name: 'Walker'),
    (Value: lsJumper;          Name: 'Jumper'),
    (Value: lsDigger;          Name: 'Digger'),
    (Value: lsClimber;         Name: 'Climber'),
    (Value: lsDrowner;         Name: 'Drowner'),
    (Value: lsHoister;         Name: 'Hoister'),
    (Value: lsBuilder;         Name: 'Builder'),
    (Value: lsBasher;          Name: 'Basher'),
    (Value: lsMiner;           Name: 'Miner'),
    (Value: lsFaller;          Name: 'Faller'),
    (Value: lsFloater;         Name: 'Floater'),
    (Value: lsSplatter;        Name: 'Splatter'),
    (Value: lsExiter;          Name: 'Exiter'),
    (Value: lsVaporizer;       Name: 'Vaporizer'),
    (Value: lsBlocker;         Name: 'Blocker'),
    (Value: lsShrugger;        Name: 'Shrugger'),
    (Value: lsOhnoer;          Name: 'Ohnoer'),
    (Value: lsExploder;        Name: 'Exploder'),
    (Value: lsArcher;          Name: 'Archer'),
    (Value: lsAttractor;       Name: 'Attractor'),
    (Value: lsBallooner;       Name: 'Ballooner'),
    (Value: lsBazooker;        Name: 'Bazooker'),
    (Value: lsBomber;          Name: 'Bomber'),
    (Value: lsClubBasher;      Name: 'ClubBasher'),
    (Value: lsDiver;           Name: 'Diver'),
    (Value: lsFencer;          Name: 'Fencer'),
    (Value: lsFiller;          Name: 'Filler'),
    (Value: lsFlameThrower;    Name: 'FlameThrower'),
    (Value: lsGluePourer;      Name: 'GluePourer'),
    (Value: lsHangGlider;      Name: 'HangGlider'),
    (Value: lsHopper;          Name: 'Hopper'),
    (Value: lsIcarusWings;     Name: 'IcarusWings'),
    (Value: lsJetPack;         Name: 'JetPack'),
    (Value: lsGapJumper;       Name: 'GapJumper'),
    (Value: lsKayaker;         Name: 'Kayaker'),
    (Value: lsLaserBlaster;    Name: 'LaserBlaster'),
    (Value: lsMagicCarpet;     Name: 'MagicCarpet'),
    (Value: lsMagnoBoots;      Name: 'MagnoBoots'),
    (Value: lsMortar;          Name: 'Mortar'),
    (Value: lsParachuter;      Name: 'Parachuter'),
    (Value: lsPlanter;         Name: 'Planter'),
    (Value: lsPlatformer;      Name: 'Platformer'),
    (Value: lsPoleVaulter;     Name: 'PoleVaulter'),
    (Value: lsRockClimber;     Name: 'RockClimber'),
    (Value: lsRoller;          Name: 'Roller'),
    (Value: lsRoper;           Name: 'Roper'),
    (Value: lsRunner;          Name: 'Runner'),
    (Value: lsSandPourer;      Name: 'SandPourer'),
    (Value: lsScooper;         Name: 'Scooper'),
    (Value: lsSkimmier;        Name: 'Skimmier'),
    (Value: lsSkater;          Name: 'Skater'),
    (Value: lsSkier;           Name: 'Skier'),
    (Value: lsSlider;          Name: 'Slider'),
    (Value: lsSpearer;         Name: 'Spearer'),
    (Value: lsStacker;         Name: 'Stacker'),
    (Value: lsStomper;         Name: 'Stomper'),
    (Value: lsSuperLem;        Name: 'SuperLem'),
    (Value: lsSurfer;          Name: 'Surfer'),
    (Value: lsSwimmer;         Name: 'Swimmer'),
    (Value: lsThrower;         Name: 'Thrower'),
    (Value: lsTwister;         Name: 'Twister')
  );

Offline Clam

  • Posts: 2187
  • Smiley: :8():
    • View Profile
Re: Lemmix
« Reply #16 on: July 15, 2011, 10:29:45 PM »
I think the "jumper" for lemmings going up a small step could be renamed to "ascender", as in L++/Lix, to avoid confusion with the Jumper skill. There's also Tumbler for lemmings that get knocked away by an explosion or fall too far after jumping, and Crawler for lemmings that get stuck in walls in L2.

Additionally, there are a few extra states associated with the Runner:
- Running lemmings jump (without the use of a skill) when they reach the end of a platform. This is different to the Jumper skill, since they don't jump as far as a regular Jumper.
- Jumpers, Spearers and Throwers (and possibly some other skills as well) are carried out differently by Runners - Jumpers jump further, and Spearers/Throwers take a run-up before throwing. These could be named 'RunningJumper', 'RunningSpearer' and 'RunningThrower'.

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #17 on: July 16, 2011, 12:00:05 AM »
Ok thanks. Sometimes it's difficult to make the distinction between skill and state. I'm not that familiar with L2.
"Ascender" seems a good idea. "Runner" is maybe a state as well as a skill. Like "climbing".

Offline Simon

  • Administrator
  • Posts: 3879
    • View Profile
    • Lix
Re: Lemmix
« Reply #18 on: July 16, 2011, 12:34:20 AM »
I'm in favor of consistency with L++/Lix of course. Or does L1 call the ascender "jumper" somewhere internally? If not, then it may be worthwhile to indeed call it something not conflicting with any other game. Also note that L2 doesn't have ascenders.

Skill, state... there's actually 3 distinct things to separate: skill in the panel, permanent ability, and current activity. Climber can mean all three. By state, you probably mean what I tend to call ability. There's no commonly used word for either ability or activity, though.

-- Simon

Offline ccexplore

  • Posts: 5311
    • View Profile
Re: Leming States
« Reply #19 on: July 18, 2011, 03:29:45 AM »
I gathered the following lemmingstates for Lemmix. The states 0..127 will be reserved. The states 128..255 will be "free" for addons.

I think you already used close to half of your reserved space.  Is it really necessary to use a single byte for states?  If not maybe you can just a 2-byte value and then you can have up to 32768 reserved states and 32768 free states, which should be plenty enough.

=================

I think it's important to get a little more context into why this is coming into discussion.  Basically Eric is trying to make styles as fully customizable as possible, so part of this involves being able to customize the lemmings animations for the various actions.  And in order to do so, it's necessary to try to identify all the actions (ie. Simon's "current activity") a lemming can be involved in.  The textual name can be used in the style definition, and perhaps also to display on screen.

I think trying to cover all aspects of Lemming 2 actions at this point may be too much to handle, especially since no one (me included) have a definitive understanding of all the numerous actions possible in Lemmings 2.  (Remember that in addition to skills, you also have more types of traps and contraptions in Lemmings 2 as well, as well as additional behavior pointed out like flinging and stunning.)  Since we already have a notion of reserved states, we can just start off with doing the Lemmings 1 behavior we're most familiar with, and then gradually extend it by adding more and more from Lemmings 2 using currently unused reserved states.

It may also be good to allow animation styles defined that don't cover all possible states/actions.  The idea would be that if a level tries to use a particular animation style, and also a skill that is not fully covered by the chosen animation style, the editor will catch this and produce an error saying that the chosen animation style do not have sufficient graphics to handle the skill in question.  That way a user-created animation style (eg. Egyptian or whatever) can be usable without having to include graphics for the many numerous actions a lemmings can do, especially adding in Lemmings 2.

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #20 on: July 29, 2011, 11:50:44 PM »
Still thinking about if I have to bind a lemmingstate to the animation or decouple it....
But there's progress! I have created my first mixed style level.

Offline EricLang

  • Posts: 464
    • View Profile
Challenge ideas
« Reply #21 on: July 31, 2011, 06:54:47 PM »
Ok the new levelpack file format is ready.
Now it seemed a nice idea to me to add "challenge" packs which refer to a levelpack. The basic idea is to make challenges for an existing level.
The ideas I have in mind now are the simple ones:

- change the number of available skills
- change the time
- change the number of lemmings
- change the number of lemmings to be rescued
- change the frametime of the game

- maybe: not allowing pausing

Any ideas to add?

Offline Simon

  • Administrator
  • Posts: 3879
    • View Profile
    • Lix
Re: Lemmix
« Reply #22 on: July 31, 2011, 11:22:52 PM »
Modify the initial release rate.
Maybe an upper limit for the rate?
A free-text field to display additional requirements of the challenge author?

You shouldn't deprive the player of a tool like the pause in a singleplayer game. Such no-pause challenges will rely on honesty anyway. If you are going to do something in this direction, warn the player at some point, but let him have the last word.

For precision levels, you can consider giving the player even sharper tools -- assigning during pause, action replay, and savestates. Maybe even interactive editing of recorded solutions, but this would be extremely advanced and difficult to make usable.

-- Simon

Offline Clam

  • Posts: 2187
  • Smiley: :8():
    • View Profile
Re: Lemmix
« Reply #23 on: August 01, 2011, 12:00:48 AM »
There are a few more challenge types, some of which we've done on the challenges board:

- Add a "checkpoint" that one or more lemmings has to pass through on the way to the exit, which forces unorthodox solution methods like ceiling routes
- Limit the number of skill assignments per lemming, or the number of lemmings that can be assigned skills
- Limit skill assignment to lemmings facing left only or right only (unworkable on some levels, but interesting nonetheless)


You shouldn't deprive the player of a tool like the pause in a singleplayer game. Such no-pause challenges will rely on honesty anyway.

I think the idea here is to let players complete challenges in-game without having to send someone a replay to check. It wouldn't be too difficult to enforce the challenge rules by tweaking some mechanics, or (preferably) have the game check that you've met the challenge criteria.


Quote
For precision levels, you can consider giving the player even sharper tools -- assigning during pause

To be consistent with DOS, if you're going to allow skill assignment during pause, you need to advance by one frame after a skill assignment (if the game is paused), since the original mechanics don't allow multiple skill assignments in a single frame. Another very useful precision tool is frame advance (press some key to advance the game by a single frame).

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #24 on: August 01, 2011, 01:55:42 AM »
Thanks for the ideas so far.

Release rate can be adjusted of course.
Maximum release rate is a nice idea.
I lke the checkpoint idea! Adding rectangular area's to the challenge is doable.
Disabling pausing is probably not a good idea.
@Clam Spammer: Limiting skill assignments to the way not very difficult to make, but I doubt the value of it.
I think I will stick to the maximum of one skill assignment per frame, but assigning during pausing I will take into consideration.

Btw: I added a random 64 bit number to the levels. I tested it and there can be made several 100.000's of levels without creating duplicates.
I will probably change the levelcode to 10 characters + 2 digits.
With the random number system it is very easy to link a replay file to a level.

I saw this guy on youtube solving 30 mayhem levels within 1 hour, and got the idea of "watching a movie" with Lemmix: take a bunch of replayfiles and replay them with Lemmix including pre- and postview screens to see the results. Not very difficult to make either. Better quality than youtube :)

The new levelpackformat is quite easy to understand. It will be possible to add any number of sections and any number of levels.
Included will be some author information of course.

Offline Simon

  • Administrator
  • Posts: 3879
    • View Profile
    • Lix
Re: Lemmix
« Reply #25 on: August 01, 2011, 02:45:31 AM »
What's the intended use of that random number?

You sound like you want to hash each level, and map replays to levels using the hash value. In that case, you don't want a random number, but a deterministic hash function. This will double as a level verification in replays, since the hash cannot be arbitrary per level, while a random number is.

-- Simon

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #26 on: August 01, 2011, 09:50:44 AM »
It's not a hash function but just a random number. When opening a replay file Lemmix will search the header of each available levelpack to find the correctlevel (in case there are many levelpacks Lemmix could create an in-memory cache).
Also the random number is used for creating a levelcode.

Offline Simon

  • Administrator
  • Posts: 3879
    • View Profile
    • Lix
Re: Lemmix
« Reply #27 on: August 01, 2011, 11:05:44 AM »
Hmm, why you prefer the random number over a deterministic hash then? The hash does all of this, and even more.

The only pitfall with hashes appears to be if the user changes a level, but wants to preserve the level code. Why do you need level codes in the first place? Older replays should indeed fail due to a hash mismatch in that case.

-- Simon

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #28 on: August 01, 2011, 12:51:50 PM »
I don't see why i should make a hash function. When a level is created a random number is assigned to it.
What added value could a hashcode give? And how would you hash the level?

Offline Simon

  • Administrator
  • Posts: 3879
    • View Profile
    • Lix
Re: Lemmix
« Reply #29 on: August 01, 2011, 01:27:38 PM »
Additional advantages of hashing:
  • Level versioning, people can manually verify they're using the same level.
  • Automated consistency check that a replay indeed uses the exact level it claims. The system immediately catches different versions of the level, since the hash will change completely. Since the random number could be arbitrarily set for the level, people can circumvent the automated check. This looks like it gives more security, and indeed it does, but that isn't the main point of it. The main point is comfort -- trustable skipping of the hassle that is manual level version verification.
  • Same hash everywhere for same modification. I can tell people to set builders to 0 and record a replay -- instead of either transferring them my level file for such a trivial modification, or telling them to set builders to 0 and additionally the random number to xyz.
  • Easy way to associate replays to challenges. A challenge is a patch for a level. The hash of a challenge can then be defined to be the hash of the patched level, even if this modified level is never written to disk.
Disadvantages:
  • Breaks the possibility of setting the same random number for different versions, thus changing level codes. (need different feature for this if we want this)
IMO, we should knowingly get rid of this possibility, due to the above gains in comfort.

If you really want to preserve the possibility to assign the same password to possibly changing versions of a level, then you can let the author make up a password for each level. This has the advantages that they'll be easier to memorize, and they can involve humor. Such things are then independent from single levels and should only go into the pack's meta information.

In addition, I do not believe level passwords are good in the first place. Packs should be freely playable, since the format is human-readable anyway, and players who wish to play the pack linearly can still do so. Offer support to track progress or highscores on the levelpack's levels instead of controlling access to later levels.

Reason I'm so persistent: This design decision seems unnecessary hard to change later. If one does, then one still has to support the older way of mapping replays to levels, or drop support for those replays.

Edit: How to hash a level: Make it a large sequence of bytes, according to any fixed protocol you choose, and then run SHA-1 or SHA-256 on it. Decide what goes into the message (the to-be-hashed sequence) of a challenge-patched level. If you feel like you will be changing the challenge possibilities in the future, consider hashing the challenges separately from the levels (in contrast to benefit point #4 above), or all old replays will assert hash mismatches.

Important is that you don't save the hash of a level into the level file itself (or if you do, don't use it unchecked), since that'll break the purpose. Loading a level from disk/pack takes a bit of time anyway, so computing the hash afresh is not a bottleneck operation at this time. Saving the hash into a level is just a matter against bit corruption on loading a level (so one has a hash to check the level against, similar as a replay specifies a level hash to verify that it fits).

-- Simon

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #30 on: August 01, 2011, 02:37:03 PM »
Thanks for the post. I'll have to read it a few times...

Hmm... there you got something. If the level changes an existing replay file will not work anymore.
However *not* storing the hashkey will make things really slow. Suppose I have a levelpack with 120 levels. I don't want to load them all in memory just to generate a hashkey of it, when searching the right level for a replay file.

Offline Simon

  • Administrator
  • Posts: 3879
    • View Profile
    • Lix
Re: Lemmix
« Reply #31 on: August 01, 2011, 02:53:05 PM »
That seems like best practice indeed -- you can store the hash of each level in the pack file, to speed up the hashtable generation. The consistency check will still happen per-level, but only when a level is actually used, not when a pack containing it is cached.

The underlying principle is that pieces in the hierarchy know about the hash values of their underlings, but not their own. I.e. you merely shouldn't put the level hash into the level itself and then rely on that alone.

Edit: The hash serves two purposes, which are quick retrieval of a level, and consistency checking. For retrieval, cache as much as you can. Consistency checking should happen only as much as necessary. The principle of the paragraph before is a way to satisfy both.

-- Simon

Offline ccexplore

  • Posts: 5311
    • View Profile
Re: Lemmix
« Reply #32 on: August 01, 2011, 10:32:59 PM »
Linking a replay automatically to a level file is interesting and kind of convenient, but seems like a lot of work to solve a problem that isn't that big of a deal.  (If I'm interested in watching a replay I probably already have the level, and it's not rocket sciene to find and open the level in question.  If the replay was recorded for the wrong version of the level, it'll quickly become obvious when the replay starts going obviously wrong.)  But I guess if you're up to it, go for it then. ;)

And using SHA seems like over-engineering.  Isn't a simpler CRC enough?

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #33 on: August 01, 2011, 11:26:38 PM »
I did some tests with a 128 bit number XORING the complete memory stream of a level, which seems enough hash to me :)
Combining this with the random generated levelnumber -used as a simpel lookup- it will probably work as intended.
The 128 bit number will be used as validation, and *is* stored in the header of the level.

Opening a replay file instantly playing the correct level is a big improvement in my eyes.

Offline ccexplore

  • Posts: 5311
    • View Profile
Re: Lemmix
« Reply #34 on: August 02, 2011, 12:18:42 AM »
I did some tests with a 128 bit number XORING the complete memory stream of a level, which seems enough hash to me :)

I'm wary of using just XOR, due to its properties like x ^ 0 == x and x ^ x == 0, which seems like it can easily lead to collisions (ie. two levels hashing to the same value) even when done on 128-bit.  It might work better if you apply it to the compressed memory stream of the level, I'm not sure.

Perhaps there's some merit to using SHA after all even though it's slower and more complicated, at least it vastly reduces the likelihood of collisions (ie. two levels hashing to the same value).  CRC might be a good middle ground but I don't honestly know what the comparison is to SHA in terms of collision probability.

Combining this with the random generated levelnumber -used as a simpel lookup- it will probably work as intended.

The use of random number negated what Simon mentioned, that it won't support a situation where you can ask the user to make a trivial modification themselves (eg. change initial RR to 99) and be guaranteed that the lookup will work.  If this is acceptable then I guess it's not an issue.

Offline geoo

  • Administrator
  • Posts: 1475
    • View Profile
Re: Lemmix
« Reply #35 on: August 02, 2011, 02:05:19 AM »
I don't see any point whatsoever in having a random number when you're already using a hash. As Simon said, the hash can be used for both finding the corresponding level to a replay (assuming the replay contains the hash value) in a hash table as well as an integrity check for the level.
The random number also has the aforementioned issue that if two players have the same level independently (for instance by changing a parameter to an existing level), that the level won't necessarily have the same random number.

SHA-1 (or any other commonly used hash function) should be easy to use because there are already implementations of them available.
XORing is prone to collisions, especially if you have a well structured level format, and thus doesn't serve too well for indexing nor integrity checks I think.

The versioning problem (i.e. having two versions of the level that are essentially the same, i.e. a solution recorded for the old version also works for the newer one) can be solved by allowing an option to force a level with a replay, and optionally overwrite the hash with the new one.
Under normal circumstances, changes to an existing level are significant though, and allowing to assign it the same random number (which doesn't work with hashes) isn't desirable: a user is prone to screw up if there's an option for it, as levels are saved very frequently in the design process; and automatically detecting whether a change is significant or not is impossible.

Offline Mindless

  • Posts: 719
  • Inactive - may respond to PM.
    • View Profile
Re: Lemmix
« Reply #36 on: August 02, 2011, 04:33:51 AM »
If I were implementing it, I would use some sort of hash without any random numbers.  SHA-1 is popular enough that you ought to be able to find an implementation in your language of choice since you probably don't want to reimplement it.  We're not specifically looking for a strong hashing function here, so even CRC32 which wasn't designed as a hashing function would probably get the job done.  Or if you want to go nice and sturdy, use a Tiger hash:D

I wouldn't worry about performance either.  Hashing isn't all that expensive.  Chances are good that whatever hardware you're using can hash faster than it can read data from a rotating magnetic disk.

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #37 on: August 02, 2011, 03:48:27 PM »
I found the adler crc. not tested yet.
If a level is created and saved, the random number will be there forever. Also when you edit it, or move it into another levelpack, or save it as a separate file.
I have converted the original dos lemmings now to the new Lemmix file format and will update you guys here with the "find replay: mechanism.

There are a lot of old lemmix replay files and a lot of custom levelpacks around.
I hope I can make a good conversiontool for these.

Offline Simon

  • Administrator
  • Posts: 3879
    • View Profile
    • Lix
Re: Lemmix
« Reply #38 on: August 02, 2011, 03:52:35 PM »
<geoo> SimonN: from his post, it sounds like he's still using that fricking random number
<SimonN> Yeah. If he keeps this, he MUST be the centralized source of levelpack conversions, or people will all end up with different random numbers.
<SimonN> Centralized = BAD BAD BAD
<geoo> yeah


Hmm, if you make a conversion tool, then there must still be a centralized source of levelpack conversions, or people will end up with all different random numbers. I do not think this is feasible for the large number of levels. In addition, you might want each user to convert L1/ONML levels himself due to their copyright restrictions.

I really urge you to get rid of the random number in this prospect. Either hash deterministically, or use the old manual way of associating replays to levels.

-- Simon

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #39 on: August 02, 2011, 04:22:30 PM »
You're right. Converting will give people different random numbers. That's not good.
And what are these stupid little characters for?
Besides automatic linking you can of course choose manually as well. For several reasons.

Offline Simon

  • Administrator
  • Posts: 3879
    • View Profile
    • Lix
Re: Lemmix
« Reply #40 on: September 17, 2011, 01:37:58 PM »
Any news on this project?

-- Simon

Offline EricLang

  • Posts: 464
    • View Profile
Re: Lemmix
« Reply #41 on: September 19, 2011, 03:21:10 PM »
The news is that real life intervened - again.