kaywhyn's and Icho's Advice for Level/Pack Creators - Always Make a Solving Replay!

[kaywhyn]

Yea, I feel the need to make this, especially as it's pretty long overdue. I think I remember Icho wanted to make a topic on this from a while back as well. It's going to be a rant of sorts, so make of it what you will!

Before I go on this rant, do note that I have yet to make and release a level pack of my own (it's currently a WIP), but I've been making levels for NL for the past 4, almost 5, years. This is long enough that I can offer this sound advice for any NL level/pack creator, and that is always make a solving replay for any completed levels you make and keep a master replay set for your levels/pack! In this way, in case a player who plays your level/pack has doubts on its solvability, which is really easy to do especially on really difficult levels as a lot of the time they give the appearance of an "impossible" level, you at least have proof that the level is 100% possible and that you the author didn't make a mistake when you made the level. You would also be able to send a replay of the level if requested by the player. This practice of always making a solving replay for completed levels is one that I've been doing right from the start of my level designing career, ever since I got into level making all those 4 years ago. Yes, I know that testing your own levels can sometimes be a lot of work, but it's a really good habit! We fortunately also have the mass replay check tool, and it's always a good idea to run replays of your levels/pack through it to make sure that everything is fine on your end!

All rightie, now starts my rant. Nothing irritates me more than when I'm playing levels made by someone else that turned out to be impossible after all, especially as I have wasted a lot of time in the past playing such levels that turned out to not be solvable! Icho can also tell you from his experiences, and he can sing you a song about playing impossible levels, asking the author for a replay, only to not get one because he/she didn't have one for the level saved.

Recently, I privately pre-tested a pack that's now cancelled and encountered a level that, while it was solvable via the normal solution, it had an impossible talisman! => I wasted about an hour trying to obtain something that was made impossible by the level design. This makes me wonder if the creator somehow didn't test the level properly for the talisman and/or put that talisman by mistake, as it was the only one for the pack. At the same time, the only way I saw that the talisman would be possible is if the Lemmings were able to step up into the containment area and hence be safe as they would no longer be in danger, but instead the Lemmings turn around because the wall is too high for them to be able to ascend it, and you can't afford to let that happen since they would walk out the level side and die.

So yes, this advice extends to and includes making a solving replay for talismans as well should you include any for your level/pack! I even asked if Armani was able to obtain the talisman, and when I was supplied his replay for the level, he didn't as well. The reason for this is I know he always goes for them, so the fact that he didn't obtain the talisman either supports that the talisman is indeed impossible to obtain.

Another thing that recently occurred last week that finally got me to make this topic was when a recently registered member who had been lurking silently on the Forums for the past several years posted and inquired about two levels in another user's level packs topic that they couldn't solve because the levels seem to not provide enough skills in order to make them solvable. While I haven't yet posted in the topic that the user posted in, I silently took a look at the levels they mentioned and I too couldn't solve them => I wasted a lot of time on two levels that don't seem to be solvable at all! Once again, another case of an author not properly testing the levels!

I definitely can't emphasize enough the importance of making a solving replay for levels you make and then keeping a master replay set for your levels/pack safe somewhere on your machine. Heck, make a backup of your replays/master replay set so that in case you accidentally delete it you at least still have a copy! I can honestly say that for many of my levels that I made all those years ago that I have begun to forget some of the details of my intended solutions, and this makes me very glad that I have made solving replays for them! I myself have yet to encounter someone who has played my levels and asking me for a solving replay because they're not sure about their solvability. 

[IchoTolot]

Totally agree here! 

A confirmation from the author that the level is indeed solvable, or a replay checker result file would be enough for me as a confirmation, but yeah level and replay should always come together in pairs!

[Mindless]

This is truly a problem that spans back to the beginning.  LARSPACK, which is likely the first Customize Lemmings level pack ever published, has an unsolvable level.  Oops!  Of course, they didn't have replay files back then, but the underlying problem is the same: "Did you test it?"

This is also why I'm trying (with limited success) to get authors to submit solutions for their level packs on the Lemmings Level Database.  It's not fun playing a difficult level when you're not sure there is actually a solution.

[Simon]

The first benefit is proof of solvability.

The second benefit is that you document the intended solution.

You'll forget a few of your own intended solutions within 5-10 years. Or you pass maintainership of your pack to another community member. In both cases, your documented solution helps judge backroutes.

-- Simon

[WillLem]

The three engines I currently maintain (SuperLemmix, RetroLemmini and NeoLemmix CE) can all be hard-coded to autosave successful replays. It's already default behaviour for all, but maybe we should just go a step further and make it non-optional?

Another idea might be to require the presence of a working .nxrp file in order to load a level. On first loading the level, we run the MRC for just that level and its detected replay (this would take milliseconds and would happen silently). If it passes, we can display "Verified Solvable" on the preview screen. If it fails or the replay is missing, we display "Unverified" instead. The user can still play the level if they wish.

Better yet, adapt the .nxlv format to contain replay data. The loading procedure then reads this data directly, and tests it to see if the level (and its talismans) are solvable. Getting the data into the level file would be easy enough: each time the level is played and solved, add the data to the level file. If data exists already, overwrite it. Create new sections for talisman data. Don't overwrite if the level isn't solved.

[Armani]

afaik, the impossible talisman that kaywhyn pre-tested wasn't because the author intentionally added an impossible talisman to their level, but rather because a bug fix ended up breaking the talisman. (Previously, Lemmings were able to step up higher than 6 pixels under very specific conditions.) Fortunately, that bug was fixed before the deadline, but since the level had been created before the fix, this issue occurred.

Another idea might be to require the presence of a working .nxrp file in order to load a level. On first loading the level, we run the MRC for just that level and its detected replay (this would take milliseconds and would happen silently). If it passes, we can display "Verified Solvable" on the preview screen. If it fails or the replay is missing, we display "Unverified" instead.

To me, this sounds like: If you don't provide solution replays when publishing your level, your level will be given a "Unverified" label on the preview screen.

I don't intend to release all solutions at the same time I publish my levels. (For all my previous level packs, I've always released the solution replays after some time had passed.) So I also don't want my levels to be marked as "Unverified" just because I didn't include the solutions right away.

Better yet, adapt the .nxlv format to contain replay data.

To me, this basically means the same thing: requiring authors to provide a solution together with the level, unless the replay data is somehow encrypted.

People on the Lemmings Forum have done a great job ensuring that their published levels remain solvable. The honor system has worked well so far.(outside of the broken talisman, it's just one person so I think it's more likely on their end) I believe we should continue trusting level authors to ensure their own levels are solvable, rather than enforcing something systematically.