[SUG][NeoLemmix Level Packs] Difficulty Ratings Scale

[Proxima]

We already have a scale that is widely used: Easy - Medium - Hard - Extreme

Sure. The problem is that right now, we can't agree on what those labels should mean (WillLem is proposing a recategorisation that would assign the same labels to different difficulties), and four ranks doesn't seem to be enough (e.g. Sammings and PimoLems are both currently labelled "Medium").

And... while I was writing up a proposal, Gronkling stepped in. Mine would be pretty much the same as his, except I think "Very Hard/Extreme" should be split up into two ranks -- we could call them "Advanced" and "Extreme".

Some examples of content I would put in each rank:

Beginner - Fun/Tricky; early LPI; early GemLems
Easy - Taxing/Mayhem/Crazy; early DoveLems; most of Sammings; early Lemminas
Medium - Wicked/Havoc; late DoveLems; mid-tier PimoLems; late Lemminas
Hard - late PimoLems; LPI/Psycho; early NepsterLems

... and I won't give examples for Advanced versus Extreme because I'm not at the level where I've played very much of that

[WillLem]

Firstly, we relax and have a cup of tea. I'm on your side here even if we disagree about the details, and I don't like being shouted at

Haha Who's shouting? Bold text is just to bring attention to a particular point, whereas CAPS LOCK IS FOR SHOUTING (that's why it's called SHOUTY CAPS)! Multiple exclamation points can also indicate a stronger or more incredulous tone, but I didn't even use those!!!!!

I've revised my earlier post slightly, and I agree with the following:

Beginners really only need to know whether content is beginner-appropriate or not, so the first two ratings should be "Beginner" (suitable for complete beginners) and "Easy" (accessible to complete beginners and will begin to stretch them).

(Suggested difficulty scale)
These ratings just seem pretty much intuitive and use the most common words for difficulty levels.

while I was writing up a proposal, Gronkling stepped in. Mine would be pretty much the same as his, except I think "Very Hard/Extreme" should be split up into two ranks -- we could call them "Advanced" and "Extreme".

All agreed.

Quick discussion of this particular point, for what it's worth:

And yes, ONML is not "advanced"... I struggled on Fun 24 the first time. Lemmings is a puzzle game, and ideally all levels should be a bit of a struggle at first.

Yes, but it's worth acknowledging the degrees between struggling on Fun 24 and struggling on, say, Crazy 2. The latter is a much more specific puzzle, and whilst it's a trick we're all familiar with, it's still definitely not "Easy". On Gronkling's suggested scale, I'd probably place ONML somewhere between Medium and Hard, even in retrospect.

To move the discussion forward a bit, though, I'm happy to concede that a lot of custom packs have compressed the lower end of the scale considerably, so Gronkling's suggestion is a good step up from the one I suggested. And, Proxima's suggestion seems an ideal further refinement as well

[WillLem]

We all seem to agree then, more or less, that the following scale is appropriate:

(I've placed the various ranks of Lemmings Redux onto this suggested scale).

Beginner - brand new players - Gentle
Easy - begins to stretch the player with basic puzzles - Quirky
Medium - introduces more complex puzzles and level design concepts - Zany / Manic
Hard - expects the player to have a solid grasp of the game, puzzles become particularly challenging - Lunatic
Advanced - requires more in-depth knowledge of a wide range of solving techniques, strategies and tricks
Extreme - challenges even the most accomplished solvers

The question now becomes - on what basis should these difficulty ratings be applied to packs?

I would suggest that only the difficulty level of the final rank needs to be indicated, since it can be assumed in the vast majority of cases that the pack will work its way up to that level of difficulty from one of the lower ratings on the scale. It also signals to the player the expected solving ability required in order to fully complete the pack.

Maybe, though, it could be appropriate in some cases to indicate that a pack does specifically include a rank of tutorial levels.

Thoughts?

[namida]

I would suggest that only the difficulty level of the final rank needs to be indicated, since it can be assumed in the vast majority of cases that the pack will work its way up to that level of difficulty from one of the lower ratings on the scale

Lemmings Plus Omega II and Lemmings Plus Alpha are of similar difficulty at the end. However, they are very, very different at the start - Lemmings Plus Omega II starts off fairly easy (probably about late-Tricky difficulty for the first few levels, though it does ramp up quite quickly), while Lemmings Plus Alpha is already post-Mayhem from the very first level. The idea that only the final rank's difficulty should matter is relying on the assumption that everyone who plays the pack intends to play it the whole way through, but some people might indeed want to just play the easier, early levels in a lot of packs.

I'm personally of the opinion that it's hard to give a very meaningful measure of difficulty. The best way would be to analyze feedback, or even better, directly have discussion with the players over what the difficulty is like - but this is very tricky to do for a new pack, even if it has been tested by a varied-skill-levels team of testers. However, I'd definitely say that if there is one, it would be more useful to reflect the whole pack, or even better to give a range; rather than to only reflect "what's the hardest this pack ever gets?"

[WillLem]

some people might indeed want to just play the easier, early levels in a lot of packs
---
it would be more useful to reflect the whole pack, or even better to give a range

Agreed, I suppose a range is more useful. Also, since most packs are ranked, it does make sense to indicate where on the scale it starts, and where it finishes.

Perhaps packs which cover a smaller range of difficulty could only mention their top level. For instance, in the examples you've given it may be appropriate to list Lemmings Plus Alpha as "Extreme", whilst listing Lemmings Plus Omega II as "Medium - Extreme".

[namida]

some people might indeed want to just play the easier, early levels in a lot of packs... it would be more useful to... give a range

Quoting like this is quite misleading. You've strung together segments that are not even from the same paragraph, let alone the same sentence, here.

[Gronkling]

Ideally you'll find the easiest levels quite easy, the medium levels match your skill level and the hardest levels push you. I think if you want a single point to judge difficulty on, the average "medium" difficulty of a pack makes more sense than the most difficult. Range is probably better though.

[WillLem]

Quoting like this is quite misleading. You've strung together segments that are not even from the same paragraph, let alone the same sentence, here.

My apologies, there was no intention to mislead here at all. I've adjusted the original message to include the full quote.

[WillLem]

What do people think about the idea of representing the various stages of difficult as numbers (or letters)?

So, the above scale could be represented thus:

1 - Beginner
2 - Easy
3 - Medium
4 - Hard
5 - Advanced
6 - Extreme

That way, packs could be labelled [Difficulty: 1-4] or [Difficulty: 5-6]

I'm not sure how I feel about this idea. I think I'm suggesting it on the grounds that it would save space in the topic title whilst indicating the range of difficulty presented by the pack, but words are always more helpful, particularly if someone isn't familiar with the suggested scale.

[WillLem]

I think we need an easier way of gauging the difficulty of our own levels. Maybe something like this helps:



Obviously, this graph was done very quickly and is by no means representative of all possible levels. However, it can be a useful indicator of where your level probably is on the difficulty scale.

In the above example, if the level provides 5 skills in total, and all 5 are required in the intended solution, then it's likely to be a "Medium" difficulty level (or possibly higher).

However, if there are 35 skills in total, and only 1 is required, it's likely to be a "Beginner" level.

Again, this graph does not suit all levels. Levels which give red herring skills or skills which are of absolutely no use in completing the level are obvious anomalies, as are levels which may only require a small number (<5) of skills, but for which the solution is extremely obscure. In these cases, however, the creator is likely to be more aware of how difficult their level is.

I'd suggest taking a look at your level: if it has a total of 10 or more skills, all of which must be used in a very particular way, it's likely to be quite a hard level. 5 or more, and it's likely to be Medium difficulty.

Comments, suggestions, agreements, elaborations?


EDIT: Perhaps "necessary" assignments must be further defined as "non-trivial". So, a gap requiring 15 Builders to get across when the level only provides 15 Builders is obviously not a "Hard" level, but the assignments are trivial, so the graph no longer applies in quite the same way. Such cases must see the gap as "1 obstacle requiring N of the same skill, with N being equal to the number of that skill which is provided", redefined as 1:1 for the purposes of this graph.


EDIT 2: It occurs to me that I could be equating difficulty with lack of open-endedness. I do genuinely think that the minute a level requires a particular skillset, and only provides that exact skillset, it becomes a difficult level. Oh No! More Lemmings (from Wild onwards) is a very difficult pack for anyone who has never played it before, and most custom levels are even harder.

I think what's happening is that the community are brushing open-ended levels away as "trivial" or "beginner", and only gauging the relative difficulty of the remaining puzzle levels, starting with "Easy" as something which the creator would expect most people to be able to solve. This seems problematic, but I can't quite put my finger on why.

[namida]

I think what's happening is that the community are brushing open-ended levels away as "trivial" or "beginner", and only gauging the relative difficulty of the remaining puzzle levels, starting with "Easy" as something which the creator would expect most people to be able to solve. This seems problematic, but I can't quite put my finger on why.

Most open-ended levels are trivial or beginner difficulty. Note that the key word here is "most", not "all".

[Strato Incendus]

I appreciate this graph, WillLem, but according to that, nearly all of my smaller levels would be considered "hard", because they usually provide exactly the number of skills the intended solution requires.

[WillLem]

Most open-ended levels are trivial or beginner difficulty. Note that the key word here is "most", not "all".

Maybe. But they needn't be, with a bit of imaginative map-making.

I appreciate this graph, WillLem, but according to that, nearly all of my smaller levels would be considered "hard", because they usually provide exactly the number of skills the intended solution requires.

Exactly. All of your levels that I have played so far are at least what I would consider "Hard".

[Proxima]

Replying to posts in the Lemmings Surges topic:

WillLem is right that the currently used difficulty labels are insufficient. While there are technically six ratings in use (Beginner, Easy, Medium, Hard, Very Hard, Extreme), the last two are very rarely used, and "Hard" is used for a huge range of difficulties. It's easier than it was for novice players to find content at their level, but for mid-range players such as Strato, WillLem and myself, it's impossible to tell whether a pack claiming to be "Hard" is at or below our level or way beyond us.

That said, I'm not really sure what we can do about it. If we tried to make a standardised difficulty scale using existing packs to illustrate each rating, that doesn't work for authors who haven't played the packs in question. Theorycrafting based on "number of assignments" or anything similar doesn't really capture the essence of difficulty.

Maybe we should encourage more of the people who play packs, rather than the authors, to indicate where they believe a pack's difficulty would fall on a scale?

[Armani]

Another question that came to my mind : Imagine there is a custom level pack named "A".  A's first rank has [medium] difficulty overall and final rank has [hard] difficulty overall. But the easiest levels of A have [easy] difficulty and the hardest levels of A have [very hard] difficulty. Then between [Medium-Hard] and [Easy-Very Hard], which one is more appropriate indication?

"Hard" is used for a huge range of difficulties.

I think this is very true. Most of the 'not-easy' packs claim themselves having "hard" difficulty.

Maybe we should encourage more of the people who play packs, rather than the authors, to indicate where they believe a pack's difficulty would fall on a scale?

As I play quite a lot of packs rating themselves as "hard", I'm more than willing to share my opinion on whether the pack's difficulty is indicated properly / which pack is harder between pack A and pack B or something like that once we establish difficulty scale everyone can agree with.