Giving feedback on custom packs

[WillLem]

I think there needs to be a protocol regarding giving feedback on custom packs.

Whilst feedback is useful, welcome, and encouraged, when unchecked it can also be quite harsh and put people off wanting to create custom content. I'm sure that there are many people who have thought about making a pack, but have seen the nature of some of the feedback that gets posted and have been put off.

Similarly, I am very sure that there are people who have removed content due to particularly negative feedback following an intense period of working on that content, leaving them feeling like their hard work and time was all for nothing.

As a basic suggestion, let's say you play a pack and there are several things you don't like about it. Before taking to the keyboard to type your frustrations straight away, take a moment to see if there's anything you do like about it, and then be sure to include that in the comments.

Ideally, each bit of negative feedback given should be balanced out with a piece of positive feedback. Praise is important: it keeps people motivated to try harder and put the extra effort in to improve their content. Simply listing a pack's drawbacks is only going to make the creator feel like a failure. Giving feedback is a powerful thing, and with great power comes great responsibility: use it wisely.

Suggestions to creators

ALWAYS get your pack playtested by at least 2 people, and be prepared to take a bit of harsh criticism initially - this will help you to shape your pack into its final product.

Then, when you release your finished pack, add a poll at the top. Something simple like:

What did you think of my pack?

1 It's excellent!
2 It's good, but some things could be improved
3 It's not really my cup of tea
4 I haven't played it yet

(Or even just a star-rating system from 1-5).

That way, you can get a good idea of what people who've downloaded your pack think, and it could help to balance out any negative feedback you get initially (unless of course everyone votes 3, in which case it's definitely worth having another look at it anyway!)

A poll would also encourage those who aren't predisposed to typing paragraph after paragraph of detailed feedback to leave something for you, for better or worse. That way, if a lot of people vote 1 or 2, but someone leaves a blazingly heated comment about how awful your pack is, at least you can see not everyone feels that way, and it'll be that much easier to take the criticism on board constructively, even if it wasn't necessarily presented to you that way.

[mantha16]

I would also add that when commenting on aesthetics it would be helpful if people remembered that styles, custom terrain backgrounds etc is an art form and as such is entirely subjective.  Just because you dont like how something looks doesn't mean its bad or needs to be changed just that its not to your taste and feedback should reflect that.

[kaywhyn]

Whenever I give feedback, I tend to focus on the positive things rather than the negative. That's why it's better if one focuses on giving the constructive feedback type instead of the harsh criticism type, the latter which tends to be hurtful. I'm one of those people who rather praise people instead of admonish them for doing things incorrectly or not so well. This is something I feel like the whole world needs to do more: Focusing on praising people rather than be blind by prejudice/discrimination. Thus, I always find at least one good thing to say about something/someone daily.   

[Dullstar]

As a community, I'm noticing that while we've had a lot of discussions about what makes a level good or bad, we don't actually seem to have this information gathered up into one convenient spot new designers can look to for advice. Instead, it's spread through a bunch of pack reviews and old threads on the level design board. We could probably save a lot of pain for both creators and with a do's and don't's pinned thread, because at least half of new designers seem to get hit with some variant of "I shouldn't need to use Clear Physics Mode to play your levels" at least once. CPM is kind of a blessing and a curse: it's a powerful tool for cutting through deceptive (intentionally or otherwise) designs, and I find it quite useful for checking the exact details of trigger areas and checking objects I want to know more about (i.e. is this thing a trap like I think it is?), but when I have to use it to solve a level, it leaves me feeling kind of annoyed, and I think some designers just either don't understand why this is annoying, or don't understand why their designs cause players to reach for the CPM button in the first place. You could say it's because of aesthetics, but the problem is, once I reach for CPM, the aesthetics go away, because CPM takes all those visuals away.

Lots of designers do this, so I'm not trying to call out any specific individual here.

[mantha16]

if anyone can't be constructive in their feedback and not understand that some things are subjective and things don't always have to take a laissez faire attitude.

Surely the idea is to be supportive not dismissive

[ccexplore]

I agree it's a fair point especially for those who are new to creating levels, that the feedback should probably be coached in more encouraging tones.  It certainly helps to end on a more positive note if nothing else.

At the same time though, we need to be mindful not to also discourage people from either giving any feedback, or to make them feel like they can't be honest with their feedback.  Balance is important but from what's said so far, it feels like everyone has very different ideas of where that balance actually lies.

The poll is probably a good idea, it's certainly pretty efficient in soliciting quick feedback, as it often takes more effort to write any detail feedback whether good or bad, a poll by comparison is much less work for everyone involved.  Though even with a poll, early results especially may still not always be as positive as one might like, and worse, it's going to outside of any single person's control.  And you also don't get to learn precisely what is working and not working for people--for example if it's "not my cup of tea", which aspects specifically aren't?  The poll won't tell you.

Speaking of art, consider that many visitors strolling through a modern art museum, some are bound to make fun of one or more works they encountered.  This is the flipside to "art is subjective".  Usually the artists aren't too fazed by this though (and in some cases for some works, may even expect this).  I think just as there are ways to improve the dialog on the feedback giver's end, there are also ways on the feedback taker's end to reel in unbalanced feedback.  If you're struggling to find the positives in some feedback, challenge the feedback giver to clarify whether there's anything they do enjoy about the levels, things that they'd like to continue to see.  I'd also encourage everyone to not assume the worst in people or take things too personally.  Language is tricky, written language even more so, and being overly negative can often be unintentional.  Give people benefit of the doubt and talking things out I believe can ultimately help clear things up and reach a better understanding for everyone involved.  And learn to become more comfortable with different views.  In the end even in worst case, it's just one person's feedback no matter what.  Someone may never like fish just because, it doesn't mean fish is bad in general or that you should stop cooking fish for everyone else either.

[mantha16]

I don't think that the onus should be on the receiver to challenge feedback.

Its called constructive criticism for a reason.

I honestly don't care what people think of my aesthetics because that is subjective and i like chaos a lot of people like order thats fine.  It's ok to not like the same thing thats cool.

For me its like if you don't like certain things you don't have to see them.  Like I don't like jump scares in movies, I don't find horror movies enjoyable but I don't demand they should cut the gore or make them less scary I just don't watch them. 

I like the idea of their being a generally accepted list of things to bear in mind when creating levels/packs.  Having a you MUST do this or you MUST NOT do that is more problematic to me.  If you have an open medium that allows creative freedom then it should be as open as possible.

like regarding CPM if you said "please bear in mind if the way to play the level requires CPM use then some players may not enjoy it" thats fine if you said "you HAVE to be able to play a level without using CPM" I find that a little bit over the top

[Simon]

I assume this sparked the discussion: Dullstar's feedback to Telelems

This frankness is exactly what I prefer to hear from others about my work: Direct focus on problems. I wouldn't want replies to me pampered merely to comply to feedback rules. I don't see it as rude either. Especially the sprite problem isn't even a problem of the pack entirely, it's also an issue in the engine.

Can't advise how to treat whoever doesn't appreciate frank discussion. It's delicate. We all would like the hobby to grow, super-thick skin shouldn't be required.

Community expectations about level design need exposure, that's right. They're always under debate anyway, whether they're more exposed or not.

-- Simon

[mantha16]

again just thats cos its what you prefer doesn't mean I am wrong for feeling differently.  and I'm sorry but you don't know how thick my skin is.  Feedback should be constructive thats not being delicate thats just what the purpose of feedback should be.

the phrase is CONSTRUCTIVE criticism  not just criticism and there was nothing constructive or helpful about what Dullstar said.

I didn't agree with a lot of what WillLem said either but I'm not annoyed about that because he offered constructive feedback.

[Simon]

Everything what Dullstar wrote was actionable, thus constructive. Feel free to ask if it's not clear how to fix the issues. Feel free to disagree that something is an issue.

Steel/earth distinction is perhaps the least actionable; it's not necessarily clear which exact tiles the problem are. In a pinch, you can ask more people to test.

-- Simon

[mantha16]

that assumes I agree everything he said is an issue and no it wasnt constructive but tbh im over this discussion and I not longer wish to be part of it so

peace out

[IchoTolot]

As long as there are no direct isults in it, I think feedback must be honest and directly pointing at problems.

Just writing: "This is great!" over and over only gets you so far. No, it must be pointed out what works and doesn't work. So that you can see which elements are worthwhile keeping and what should be considered to adjust.

Back when I released my first pack (Lemmings Reunion), Nepster gave me a ton of feedback. Some of it was harsh, but to the point. It helped me to figure out the bad parts and I adjusted a lot of things because of it. He also wrote what worked well and that gave me hints on which things should be focused on more.
Removal of timers, unnessesary precision, making some traps more visible and reducing of redundant elements (too many builders) where his most common criticisms back then. I learned my lessons and my content got better because of it.

Feedback must reflect the true feelings of the player, otherwise it is nearly worthless and can be even misleading. When reading it the creator does not need to adjust everything pointed out, it's up to them what lessons are to be learned and it's ok to disagree with some of the comments.

In my opinion, the most important lesson here is:
People need to learn to free themselves from feelings while reading feedback and focus more on logic. Feelings can often get in the way of thinking clearly and wrong impulsive decisions are being made because of it. Take a deep breath, take a step back and analyse in peace what information can be exacted from your feedback.

[mantha16]

IT WASNT CONSTRUCTIVE

[kaywhyn]

As others have already said, it's just one person's opinion. Thus, don't take these things personally. Instead, these should be looked at as ways to improve. Perhaps it's best to look at them as suggestions instead. Of course, as the level pack author, you don't have to follow them. You can choose to follow all, some, or even none of them. It's also ok if your first pack didn't turn out as well as you wanted it to. Many of the members here had first packs that were ok but they would be a lot better if this and that changed, and these authors have improved tremendously since with every subsequent pack they have released. This is all a learning process. I can't say the same for myself, as I have never made one, and I likely don't intend to, as I am no good when it comes to making levels and am much better at solving.

Also, keep in mind that it's impossible to please everyone. Just look at the various topics in the discussions/suggestions board. Some issues were very easy to resolve, while others had a very lengthy discussion before any action/changes were made. The best you can do, in most cases, is tailor your pack to some, but not all. Of course, don't disregard your own preferences as well. In the same way that art is subjective, there will always be levels that some people like but others won't. Yes, there are some levels from L1 and ONML that were generally liked or hated by most, but there are some who thought otherwise. This is very true of the difficulty of levels. I have struggled on levels that some found easy or vice versa, where there are hard levels that most have struggled with but I didn't find so bad.

[mantha16]

the only thing i have to say is it wasnt my first pack but the feedback i got from sammings I did put into effect where I agreed with it.  I am largely really pleased with the aesthetics and IF I do decide to make changes and re-publish people should not expect a lot of aesthetic changes other than trying to better define backgrounds v terrain and steel v non steel. 

And once I understand the technical issues in changing the recolourings for athletes I will change that but only that (and correcting the actual errors) 100% I will not be redesigning the sprites because I love them

[kaywhyn]

Ah yes, I totally forgot about sammings. Well, that's good. You're well on your way. Don't give up and keep trying.

[ccexplore]

It seems like there's an unfortunate misunderstanding at play with the recent happenings here.  I've reread Dullstar's review a number of times, and as far as I can tell, the only point made during the review about the sprites specifically, was that neutrals and athletes look exactly the same as regular lemmings when usually they are colored differently, and it is helpful, even vital, to many people to be able to clearly tell which lemming is which kind when they can all appear at the level at the same time.  This is no different from board games like chess and checkers using different colors to depict which player a gamepiece belongs to.  I don't see this as somehow translating to a demand to redesign the current sprites completely.  I do think it's maybe a bit unfortunate that so many sentences were spent during the review on that one problem that is an understandable oversight.

Dullstar did then mention in another topic the possibility of an in-game option that would allow playing levels using the default sprites instead of custom ones.  The primary intention is to deal with cases where the custom sprites present problems for certain players such as difficulty telling the different kinds of lemmings apart.  It was effectively a proposed workaround and not even a particularly agreed-upon one--WillLem quickly suggested two other ways the situation can be handled without such a proposed option.  It'd be like if you got a chess set where all the pieces are one color, and people are talking about the best way in that case and other similar circumstances, to tweak the pieces slightly so that you can actually tell which piece belongs to which player.

I think where things got a bit off-track was when Dullstar also started talking about cases where he worry sprites being designed to be misleading.  But from past context, I believe the intent there is only talking about that possibility in general/in theory, and not meant to be directed specifically at the levels he reviewed earlier.  If anything that was more a reference to WillLem for example, who had actually made a little bit of a splash in the still-recent past of exploring uses of misleading/"unfair" visuals as part of the experience of playing a NeoLemmix level.  For better or worse, some people here gets very worried about misleading visuals being used in level designs (even if in some cases only as a theoretical possibility, rather than having actual examples out there already), and sometimes this kind of concern gets incorporated into the details of various suggestions for improving the NeoLemmix player and/or editor.

I can see how taken all together, the various posts Dullstar made within the same time frame, could unfortunately be not too difficult to get misread as some kind of directed passive-aggressive dig.  But it is not the actual intention I think.  I hope one can look past all this and not become too discouraged by a single person's review, opinions and reactions.

[grams88]

Hope everyone is doing okay.

To be honest I've not found one bad pack here and the amount of work that goes into making a level pack can be a lot of work, it might feel more work for others, a lot of people work at different speeds. I don't like to criticise peoples work especially as we are all unique and like different things. I was in the duke3d community and I felt like they were being very harsh about the earlier duke maps, I love those earlier duke maps, it's just who I am. I don't like to criticise others after all we all put in hard work. I would say try not to criticise in a sense but do try to inspire others, maybe use terms like I love that you did that with your level but.

I'm hoping no one takes things too personally as that creates a hostile environment. Sometimes people can take things personally and maybe it can lead to people just giving up on it altogether and not coming back to the forum. I feel that would be a sad state of affairs just because they felt like they were being overly criticised. IT probably happens on a lot of video game forums where someone feels like everyone is ganging up on them. This can feel very real to people. I don't like seeing others upset over things like that especially as things could of been worded a bit different. Funny thing was I talking to my brother about this very thing and he was thinking the same as myself, if we were to be critical try not to overwhelmed the person who created it. We can't find someone guilty just based on opinions we need the facts to the case.

If I was to criticise I feel like I would say something like this, I stink at timed bombers, I see a lot of these levels have timed bombers which might be my devil.

I've had my fair share of criticism for my level packs, I'm hoping the nuked lems does 100 times better than the skii sloping lemmings. So far people have been liking the pack. Makes me feel good about myself.

[mantha16]

i appreciate the thoughtful replay CC you have at least tried to see things from my point of view.    i do think people need to consider tact and that not everyone appreciates a blunt approach and even if the majority do that doesnt mean its ok to assume that everyone wants it that way. 

whilst its not just about a combination of things I could list exactly what i object to but at this point i think to do that will inflame things further and so it counter productive.  I would simply ask that in future if you want to criticise my stuff in future thats cool but at least if you are going to actually give some suggestions and make it seem like it is a suggestion rather than a demand.

[mantha16]

also gram whilst i couldnt do a lot of ski slopings cos it was very very difficult i did actually like it more than a lot of people seemed to

[WillLem]

As a community, I'm noticing that while we've had a lot of discussions about what makes a level good or bad, we don't actually seem to have this information gathered up into one convenient spot new designers can look to for advice.

I think I suggested that we should have a topic for this as well... can't remember where I suggested it, but it definitely seems like it'd be a good idea. That way, new content creators can simply be directed to the guidelines rather than people having to give feedback which could be taken the wrong way.

That said, it's important to have ongoing discussions about these things, and I wouldn't want such a topic to detract from feedback altogether. Players should still post feedback in topics, and discuss exactly which part of the guidelines the creator may need to refer to. A set of guidelines simply shows that these are things that have been discussed and considered previously, and so are less likely to be taken personally.

At the same time though, we need to be mindful not to also discourage people from either giving any feedback, or to make them feel like they can't be honest with their feedback.

You're absolutely right, and - again - I don't think anybody wants to see less feedback.

I just think it's worth considering that if a newcomer has released their first or second pack, and the first few comments they get are full of problem-highlighting and general naysaying, this is likely to have an effect which is ultimately negative for everyone: the creator will feel discouraged, and others who may be considering making packs and taking part in forum discussion may also not want to get involved. Thus there are less players, less creators, less content, less learning, and the same problems and frustrations keep recurring time and time again.

The solution is simply to temper any negative feedback or problem-highlighting with a piece of equally-weighted positive feedback, at least for newcomers and people who have shown that they respond better to praise. I have to say, I think that generally, people on this forum do do this. But occasionally it does feel like the emphasis is on the problems.

Having a you MUST do this or you MUST NOT do that is more problematic to me. If you have an open medium that allows creative freedom then it should be as open as possible... like regarding CPM if you said "please bear in mind if the way to play the level requires CPM use then some players may not enjoy it" thats fine if you said "you HAVE to be able to play a level without using CPM" I find that a little bit over the top

Agreed. Guidelines should be exactly that - guidelines, which creators can choose to ignore. The understanding would of course be that content which ignores the guidelines is not guaranteed an audience. Some creators will be OK with that, but others may not. Either way, being informed gives everyone the choice, and everyone knows what to expect.

it must be pointed out what works and doesn't work... Feedback must reflect the true feelings of the player, otherwise it is nearly worthless and can be even misleading

We agree on this, then: pointing at problems should be balanced out with praise for what's good, and vice versa. Those who naturally see the good in things should learn to be more honest about what they don't like, and those that tend to see the problems should learn to take a step back and look for the positives before heading straight for the comments.

People need to learn to free themselves from feelings while reading feedback and focus more on logic. Feelings can often get in the way of thinking clearly and wrong impulsive decisions are being made because of it. Take a deep breath, take a step back and analyse in peace what information can be exacted from your feedback.

I do agree with this, and it's an important point, for sure. Those of us who tend to get heated easily could certainly learn to chill a bit now and then. However - similarly, perhaps those with cool-as-steel logical minds could do with practicing warmth and sensitivity now and again, as well!

I don't like seeing others upset over things like that especially as things could of been worded a bit different.

We all have our roles to play in a community forum. Every forum needs its grams88 - someone who wants to see people happy and co-operating.

Incidentally, I'll be looking at Nuked Lemmings for the LP series, I'm looking forward to it!