Lemmings Forums

Lemmings Boards => Contests => Topic started by: namida on March 01, 2015, 01:03:12 AM

Title: Global rule debates, what engines to allow
Post by: namida on March 01, 2015, 01:03:12 AM
Hey. So, we've run two of these contests so far, and about to start a 3rd; I'd like to have some discussion with people over what they think of the format, prizes, etc so far to take into account for future contests.

Feel free to give any input, but in particular:


1. Is there anything in the "global rules" you don't like, or particularly do like? By this I'm referring to the rules that are (semi-)consistent across all contests, not the specific rules of each one.
2. What do you think about the specific criteria that have been chosen for the contests so far?
3. How do you feel about the prizes? Are there any you feel shouldn't be options, or anything extra you'd like to see?
4. How easy do you find it to understand the criteria's rules?
5. If you haven't participated so far (but have been following them), is there any specific reason for this, or is it just that you don't have the time to enter and/or don't feel up to making a good enough level?
6. Do you think a long enough timeframe is given for voting? Too long? Not long enough?

Keep in mind that there are two basic aims with me running these - the first of all is to provide fun for the community, but the second reason is also to encourage some more new level designs, which I feel has been a bit on the slow side lately (or maybe it's just me).

Let me know what you think and I'll take it into account for future contests (within reason of course; I'm not going to start offering US$100 as a prize :P).
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Wafflem on March 01, 2015, 06:20:33 AM
1. All the global rules are fine, except in the Playing phase where others are allowed to discuss the levels, but then you can't talk about them when voting. I feel that discussing the levels during the Playing phase makes it also biased and also influences the results, and I feel they should be discussed after the contest is over. The sharing replays should also wait.

2. I like the specific criteria for each contest. I like that in the first contest that you are supposed to use an unofficial tileset, it (finally) got other authors to use your Lemmings Plus tilesets. I also agree that the second contest was generally accessible for everyone due to it being 20-of-everything.

3. All the prizes are fine as is, although there aren't enough options. Some ideas for prizes:
-If you are creating a new fangame project, the prize can be to show the winner a "sneak peek" of the pack by PM.
-If you have created a new style for a new pack, and if you have finished the style but have not completed your pack yet, the prize can be the style so the winner will have the opportunity to create levels with it.

4. In general, the criteria's rules were very understandable.

6. 3 days is fine for voting, since I don't think a lot of people played the levels in the second contests. We only got about 5-6 votes.

I also agree that there hasn't been much level design lately, aside from IchoTolot's upcoming Lemmini levelpack. There hasn't even been any NeoLemmix levels that uses the objects, gimmicks, etc. (except for my packs, of course)
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 01, 2015, 04:30:52 PM
One thing I'm thinking of in future. To avoid complicating things, any monetary prizes will just be listed by their value. I'll put in the general rules that monetary prizes may be claimed via (these are mostly the same options that already exist):

> PayPal
> Bitcoin (or a similarly-established cryptocurrency, eg. Litecoin or Dogecoin)
> Direct purchase from (or donation to) an online service, subject to them accepting PayPal (or, if highly reputable, directly accepting credit/debit cards and/or Bitcoin)
> Direct bank transfer to a New Zealand bank only

I'm also going to stop offering the advertising option for a while as two people have won it and have it on hold for future use; I don't want to have collisions of too many people wanting to use it at the same time, so I won't offer it for a while. Don't worry though, I have something that should be fairly appealing that I plan to offer instead.

The "set the next contest's rules" option will probably almost always be there, as this is a good fallback option for those who are interested in the contest but don't want to (or can't) claim any of the other prizes.
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 01, 2015, 08:06:26 PM
Another thing I'm considering, to hopefully increase participation (though maybe at the risk of reducing the overall quality of levels, so I'd have to give this a trial run) is getting rid of the 2nd place prize, and instead awarding a prize to one participant (other than the winner) selected completely at random. This way, even those who aren't confident about being able to win will have a reason to enter (and maybe even surprise themself with how their level turns out in the process; recall that Crane did not at all expect to win the first contest yet he did!).
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Wafflem on March 01, 2015, 08:14:42 PM
That's an interesting idea. I'd like to see how this would turn out for the next contest. Hopefully this will give some incentive for people to make levels.

Of course, it will also depend on the prizes that you will give out.
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Nepster on March 01, 2015, 08:52:04 PM
Who (of the participants of the first two contests) was actually encouraged to participate by the prizes?
I for one am rather indifferent concerning the prizes, but simply like the challenge to create nice levels meeting certain criteria.
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 01, 2015, 09:00:07 PM
That's an interesting idea. I'd like to see how this would turn out for the next contest. Hopefully this will give some incentive for people to make levels.

Of course, it will also depend on the prizes that you will give out.

If we go ahead with the tag-team idea, then I won't be able to do the random prize for that one since there'll effectively be two first places. :P It'll already need some slight modification to the prize structure...
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: IchoTolot on March 01, 2015, 09:17:23 PM
For me the prizes are absolutely secondary (altough I am very happy with the advertisemend prize which I will claim in the future  :laugh:). I am new to the designing community and I want to show and also improve my desining skills with the feedback I get here (and fulfilling the criteria is also a lot of fun). Furthermore I get to see more levels from other people (and getting to know "their style"), which might give me more ideas for my own levels and as I love solving them it is just more Lemmings to play for me  :).
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: mobius on March 01, 2015, 10:45:22 PM
So far I think the contests have been good. I can't think of anything really to improve. [I mention something below] I have not much time lately to get involved and have been doing other things besides Lemmings anyway.
Over the past months I've been seeing or getting messages from people on you tube who are not on this forum who seem interested in Lemmings. I've tried to bring them aboard but with not much luck.

The prizes aren't real important for me either. I think in my life I've only ever entered contests because the "journey" was (or I thought was going to be) fun. I never did it for the prize.

I don't really like leaving out the whole backroute fixing phase. I liked that about the old contests (the one geoo started). This way the author can at least offer a somewhat patched up version for the final voting. Backroutes, even if minor can sway an opinion of a level a lot.
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 02, 2015, 03:19:12 PM
^ I've put up a poll to see how people feel about that. It does need to be taken into account that your suggestion there, and DynaLem's "don't allow discussion even during playing phase" are more or less mutually exclusive, though I guess that any replays could be PM'd to the creators.

(Don't take this the wrong way; it's not saying that I've decided for sure we're implementing that playing phase rule. Just pointing out that the two are somewhat incompatible.)
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Simon on March 02, 2015, 03:32:43 PM
The open-source way, which seems the only proper way to do this <_<;;, is to encourage a fast release cycle.

The author could in theory release a non-solvable version, but I don't feel he ever wants to. Heartily encourage the authors to send a validating replay accompanying a level change, but don't require it.

-- Simon
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 02, 2015, 08:29:40 PM
The replay thing was more to ensure the author hadn't changed the intended solution, just patched up unintended ones.
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Nepster on March 03, 2015, 04:36:54 PM
Quote
Timeframe:
1st Phase entries close 12th of March
2nd Phase entry close date is TBA, it will be at least 7 days after the 2nd phase begins
How does everyone feel about this? :lix-unsure:
I found the two weeks for contest 2 already rather tight, but now one has just 10 days (and only one weekend) for phase 1 and certainly less for phase 2 (unless one checks messages every day).

One certainly can do a "Speed creation contest" (e.g. create a full level pack of 10 levels within one week), but I fear that the level quality will suffer if the time frame is too tight.
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 03, 2015, 05:48:39 PM
I could extend it if it seems problematic. I just felt it could be a bit shorter since in a way, each entrant is only designing half a level each. I'll put a poll up, and extend it if most people think it should be.
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 06, 2015, 04:34:13 AM
Alright, based on the feedback from the poll allowing changes, the developers will be allowed to revise their levels during the Playing phase. For this reason, I'll also make that phase a bit longer.

These changes will not apply until the next contest, to avoid confusion (ie: starting from contest #4). There will probably also be some limitations on the changes; perhaps along the lines of that there will be one update allowed in the middle of the playing phase, and another allowed at the end of it - not endless revisions. However, this is open for discussion if anyone feels that this isn't sufficient. For obvious reasons, it also might not apply in contests where backroutes are not really relevant - for example, if there was a "make the best looking level, solution doesn't matter beyond that the level must be possible" contest.

I'd also like to discuss something else - the contests so far have allowed Lemmix (and by extension, DOS Lemmings, since these two are pretty much compatible with each other), NeoLemmix, Lemmini and SuperLemmini (or some subset of those four). Where possible, how do people feel about also accepting Lix in future contests?

(I probably won't expand the selection any further beyond that; but since Lix has just as significant, if not more so, of a share of usage as the other four, I think it's worth considering including it.)
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Wafflem on March 08, 2015, 01:50:06 AM
I'd also like to discuss something else - the contests so far have allowed Lemmix (and by extension, DOS Lemmings, since these two are pretty much compatible with each other), NeoLemmix, Lemmini and SuperLemmini (or some subset of those four). Where possible, how do people feel about also accepting Lix in future contests?

There should be a poll on that. One thing about Lix is that you can play Lemmix and Lemmini levels on there, as long as you place the files of the official styles in the Lix folder. Another consideration is that not everyone has played Lix, so they may not be familiar with the Lix skills like the runner and jumper, bombers that blast back Lix that are in the vicinity of the blast radius, and mechanics such as pixel-perfect steel areas and Lix fainting for a brief moment when falling down from a certain height.
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 08, 2015, 03:27:37 AM
The mechanics differences can be said about any engine, really. NeoLemmix and SuperLemmini also have the pixel-perfect steel areas (heck, due to autosteel they don't even have to be straight-edged; no idea if Lix is the same in this regard), and NeoLemmix also has extra skills.

I wasn't aware Lix can play Lemmini levels, actually. It currently can't handle NeoLemmix levels, not sure about SuperLemmini.

Let's put a poll up, then...
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Wafflem on March 08, 2015, 03:50:23 AM
I saw that Proxima added the DoveLems (a Lemmini-based Fangame) levels to Lix, so Lemmini levels are indeed playable on Lix.

Some things to consider:
Even if you play the Lemmix/Lemmini levels directly on Lix, the intended solution may be affected, due to that Lix faint before getting up when falling from a certain height, and that Lix does not have one-way arrows.
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Simon on March 08, 2015, 11:44:27 AM
For contests, you should play each level on its author's game. You want comparable replays.

Loading levels from other games is purely for convenience. That has no impact here.

Whenever you broaden the set of allowed games, you require all entrants to become familiar with all games.

-- Simon
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 16, 2015, 12:06:34 PM
Alright so, given the relative lack of interest in this contest - which may just be the format, mind you - I've thought about revising how closing entries should work for future contests.

Just to be clear, this is just an idea at this point. Whether it becomes the new system will depend on feedback, of course.

My idea is to have the entries closing based on a few different criteria, something along the following lines:
1. Entries will never close sooner than *insert date here* (generally about a week from the opening of the contest), overriding all other rules
2. Entries close immediately when X number of entries have been received
3. If at least (X / 2) entries have been received, and no further entries or revisions have been submitted for one week, entries will close
4. Regardless of number of entries, entries will close on *insert date here* (I'm thinking about 2 months from the opening of the contest; this is just a failsafe in case even (X / 2) isn't reached)

Obviously, there'll still be some absolute minimum that applies no matter what. My first thought is to make this "at least 3 entries", but a possible alternative is "at least 2 entries, with second place only receiving prizes if there are at least 3 entries".
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Nepster on March 16, 2015, 04:32:25 PM
Alright so, given the relative lack of interest in this contest - which may just be the format, mind you - I've thought about revising how closing entries should work for future contests.
In my case, it is purely the time frame that prevents me from participating. As mentioned before, I simply don't want to destroy another ones efforts, because I didn't find the time to create a proper level and therefore had to hand in some mediocre terrain/objects+skills.

Quote
2. Entries close immediately when X number of entries have been received
I am totally against this, as it defeats the intent of level contests: We do want to have as many participants (and new levels) as possible, so excluding someone just because several others were faster is nonsense. (Sorry for the strong words, but I am really passionate about not having this rule).

I have no real preferences among the other rules, but would like to keep the rule as simple as possible. I could actually live with a slight modification of the current rule like "3 weeks + 2 additional weeks if less than 4 participants".

PS @namida: Why haven't we seen any level of yours in the level contests so far?
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 16, 2015, 05:46:34 PM
You raise a good point there. My thought was setting said "X" higher than the number of entries we usually get, but perhaps a better idea is to scratch that one, and just have the "one week (or some other time frame) since last new / updated entry", as well as a longer-than-nessecary "final" deadline.

As for none of my levels - well, obviously the rules don't prohibit me from entering (only from getting prizes), so I could do so. Due to the shared-winner nature of this contest I probably won't, but you're right, I should probably enter levels in future ones just for the fun of it. :)
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Nepster on March 16, 2015, 06:15:39 PM
As for none of my levels - well, obviously the rules don't prohibit me from entering (only from getting prizes), so I could do so. Due to the shared-winner nature of this contest I probably won't, but you're right, I should probably enter levels in future ones just for the fun of it. :)
I don't get the point of "contest host cannot get prizes": With the topic of the contest chosen by preferences of forum members and the winner chosen by poll, there are not much possibilities for rigging a contest (well, at least not more than for any other participant :P).
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 16, 2015, 06:22:05 PM
Theoretically, I could just edit the poll results through the database. Of course, it should be quite clear by now that I wouldn't do such a thing, but still. :P Part of that is also the fact that there's little point in me giving a prize to myself. :P Perhaps the only one that may stand out (since on principle I won't just do it "because I can") is when the advertising option is offered; obviously I can't really "give myself" $10; I get to pick the contest rules anyway (if no one else has that as a prize), and I can add something to NeoLemmix any time I feel like it. xD

So, in a way, it's kind of so that two people (or however many) still get prizes, I guess?
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Nepster on March 16, 2015, 07:15:21 PM
And I could create ten new accounts myself and let them vote for my own level ;P.
... obviously I can't really "give myself" $10; I get to pick the contest rules anyway ...
But you can prevent having to give $10 to someone else ;). And up to now, you picked the contest rules, that people showed the most interest in. So theoretically you could use this prize to choose an otherwise absolutely unpopular topic without making others mad at you...
So, in a way, it's kind of so that two people (or however many) still get prizes, I guess?
No, prizes should go to the most popular levels. So if you get one of the prizes yourself, this should be seen by others as an incentive to learn what made your level better and then try harder next time. And the prizes don't seem to be very important to people here anyway.
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on March 17, 2015, 03:48:38 AM
Quote
And I could create ten new accounts myself and let them vote for my own level ;P.

Yes, but that'd be more noticable when there's suddenly 10 accounts doing nothing but voting for the same level. :P Assuming that something gave me reason to go digging into the database to check their level voting habits, that is. xD

Quote
But you can prevent having to give $10 to someone else ;). And up to now, you picked the contest rules, that people showed the most interest in. So theoretically you could use this prize to choose an otherwise absolutely unpopular topic without making others mad at you...

Those are valid points, I guess...
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Wafflem on March 17, 2015, 04:17:49 PM
And the prizes don't seem to be very important to people here anyway.

The $10 and the new NeoLemmix gimmicks are important to me, mainly because I don't want the two gimmick slots in the NeoLemmix Editor to be left blank should NeoLemmix ever be finalized, and the $10 could be used for more important things like donating to charity (which I did for Contest #2).
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Wafflem on April 07, 2015, 06:31:46 PM
Since you are the one who uploads the levels in the database on your website, when a contest is currently running, is it okay if I send a level pack to you with the contest level in advance (best during the Playing/Voting stage), but then you won't upload it until after the contest is over?
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on April 07, 2015, 06:40:45 PM
I'm fine with that.
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on April 07, 2015, 07:06:48 PM
Alright, some things I'm considering for the next contest (and possibly future ones, depending on how well it works out), based on existing feedback etc - feel free to discuss these, as they aren't "finalized" until the contest begins.

1. Lix
If there is nothing inherent in the criteria that would prevent a Lix level from meeting it, single-player Lix levels will be allowed in addition to Lemmix / NeoLemmix / Lemmini / SuperLemmini. In the case that rules make reference to the "original 8 skills" in any way, the non-flinging bomber is the one that counts as part of this. Unless contest rules state otherwise, Lix-exclusive features/skills are permitted. Multi-player Lix levels are not permitted, unless the contest rules specifically state otherwise.

2. Revision of levels
Revised version of the levels will be allowed during the playing phase, and the playing phase will most likely be extended to account for this. How this will work is that during the playing phase,  there'll be several extra "deadlines", and at each of these, if updated levels have been received, they'll be posted (along with an announcement that updated versions are available). You can submit as many updates as you like until the final update deadline has passed (which will be at least a few days before the voting phase starts); the newest one prior to the deadline will be uploaded as each deadline passes. Once the final deadline has passed, no further updates can be submitted - the level will have to enter the voting phase as-is, though as usual you're free to update it again once the contest has ended.

3. Custom graphics, VGASPECs
This is more just a change in formatting than anything else; but from now on, it will not be explicitly mentioned that these are allowed. Instead, it will only be explicitly mentioned if they're not allowed - if nothing is mentioned about them, assume they're fine. The rule still stands that if using a custom graphic set, you must provide a copy of it with your entry, unless it's included by default with the engine (eg. if you make a NeoLemmix level using one of the Lemmings Plus graphic sets, you don't need to include that, as NeoLemmix includes the LP sets by default).

4. Closing dates
There'll be a minimum closing date, which will be a similar timeframe to in the past (about two weeks). However, on top of this, even after this date passes, entries will not close until 72 hours since the last entry was received (or one week since the last entry was received, if there are less than four entries so far).

5. Minimum number of entries
Entry period will no longer extend if the contest has not reached four entries. The contest will immediately be cancelled if only one or two entries have been received. If three entries have been received however, it will proceed as normal, except only first place will get a prize (instead of both first and second).
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: Nepster on April 16, 2015, 05:30:53 PM
One comment on the layout of the first post of a newly started Lemmings contest:
The general rules (which are in general not terribly interesting) are extremely prominent at the moment, especially as they take up the first half of such a post. However the most important part of the post is the theme/design goal. So I would like to suggest to swap the general rules with the contest-specific information in the future. Perhaps one could even hide the general rules inside a spoiler-tag?
Title: Re: Official Level Design Contests - Rules, Prizes, etc
Post by: namida on April 16, 2015, 10:10:33 PM
I would prefer to keep them at the start, but putting them inside a collapsible spoiler tag sounds like a good idea. :)
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on April 29, 2015, 05:26:11 PM
Okay so, I think it's seeming like allowing Lix is a good idea - it's not really that hard to get used to it (as someone who's barely used Lix before, I had little trouble handling the Lix entry in the latest contest).

Something I would like further input on is the rules around updating levels during the playing phase. As most of you are probably aware, the previous rule was that once entries close levels can no longer be updated for the contest. The latest contest, the rules have changed to have certain "update points" - essentially, extra deadlines by which updated versions may be submitted.

Some previous user-run contests have simply allowed as much updating as the entrants like during the playing phase, and it goes without saying that this option has been proposed and considered for these official contests too.

Advantages of doing it this way would include potentially better-quality final entries, as well as more flexibility for those who have trouble meeting set deadlines. Disdvantages would include that people with more spare time would potentially be able to improve their level more, as well as the constant question of whether a level is in its final form which may be a deterrent to playing it earlier on; as well as that people may need to look through the topic to find the newest versions of levels if the entrants post them themself.

I'd like some input on whether to keep this as it currently is; go back to the original "no updates at all" structure; or something else instead. One possible alternative I've considered is allowing updates whenever the entrants feel like it, up to a certain cutoff point after which no further updates can be made. Another option is a hybrid between this and the current method - have a cutoff point up to which the entrants may post updated versions, then after this also have just one further update point using the current structure (submit via PM, and all updates are posted at the same time).

What are your thoughts on how this should be handled, everyone?
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: Simon on April 29, 2015, 05:43:06 PM
I feel like backroute fixes should happen as free-form as possible. Have the entrants post updates to the contest thread, without the indirection via PM. When people fix something in their project, they want it out ASAP.

The host can fish out the most recent versions, and collect them in the summary post.

If necessary, freeze versions once voting starts.

-- Simon
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on April 29, 2015, 05:44:24 PM
I would rather freeze them at least a few days before voting starts, to give people time to play the finalized versions (without worry of further updates appearing) before the voting period. Free-form up until such a deadline is an option that can be considered.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: GigaLem on April 29, 2015, 07:17:00 PM
Maybe we can make a more major contest or something like the idea i had a few months ago
and possibly the ability to put in more than 1 submission at a time
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: Nepster on April 29, 2015, 09:13:23 PM
I think, a bigger contest makes only sense if there will be more participants than in the contests so far. Otherwise we will likely end up with only two or three entries in each category.

Concerning backroute-fixes during contests, I second Simon's suggestion.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on May 23, 2015, 02:39:54 PM
Alright, so it seems that unlimited updates is preferred.

How's this sound regarding future structure:

- Creation phase - Keep it exactly as is (ie: levels are submitted privately, and all released at the same time when the next phase starts)
- Playing phase, Part 1 - Up until a certain point, creators are free to post updates whenever they feel nessecary; for the sake of tidiness we may have a seperate "updates" topic which is only for posting updated versions of the levels, seperate from the main playing phase topic.
- Playing phase, Part 2 - After the cutoff point, no further updates are permitted. This is to give people a chance to play the finalized level before voting phase begins. This would be shorter than Part 1 of the playing phase.
- Voting phase - This, like the creation phase, would stay the same way it currently is

One question regarding the playing phase - should we allow further entries during this time, since we're also allowing updates freely? The downside may be that fewer contests would get 3 levels before the end of the creation phase, due to people thinking "well, I'll just submit it during the playing phase so I have more time". Perhaps a compromise could be that the subission may be first made during this, but people must at least express their intent to submit a level before the creation phase closes (with the main question here becoming - what happens if someone were to intend to, but then be unable to, and thus the contest is left with fewer entries than suspected - particularly if this then causes it to not reach the required 3 levels).
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on June 10, 2015, 05:28:13 AM
Someone asked via PM whether I have any preferred naming scheme for updates. Answer is, no, it really doesn't matter - just submit them with whatever filename you keep them under. :) I only keep the most-recent copy of levels, for which I name them: [author]_[title without spaces].[lvl / ini / txt]; but honestly, it doesn't bother me at all whether you submit them this way or not - it isn't exactly hard for me to rename them, especially with the "Save As" box when downloading it. :) So - no, there are no rules (or even preferences) about the filenames of your submissions. :)
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on June 10, 2015, 02:20:32 PM
So, I'd like some further feedback on how to handle the deadlines.

The current structure is:

Initial submissions
Initially, a deadline is set, which is always midnight (UTC) on a specific date, usually about two weeks from when the rules are posted. Entries will never close before this deadline, overriding the rules below.
However, if entries are received close to the deadline, it is extended. If 3 or less entries have been received, entries close one week from the most recent entry. If 4 or more have been received, entries close 72 hours from the most recent entry.
A level being updated (prior to entries closing) also has the same effect - the deadline is extended from the time of updating by one week / 72 hours as applicable.

Updates
Updates (once entries have closed and the playing phase has begun) work more or less the same way as initial submissions.
The one difference is that the deadline extension is 72 hours, regardless of the number of entries.


What I'm wondering is whether this structure should be kept, or whether it should be changed. Possible suggestions (aside from simply staying how it is now):

- During the initial submissions, only extend the deadline in respect to new entries, not to an already-submitted entry being updated.
- Do not extend the initial submission deadline at all. However, keep the extension for updates, so that the end of the updates-allowed part of the playing phase only comes once no more updates are being made.
- Go back to absolute deadlines in all cases. Even if a submission is made one minute before the deadline, don't extend it.

I'm thinking my preference is the first option - keep extending the deadline for initial submission as long as new entries are coming in (not just updates to existing ones), and make no changes to the current rules in regards to deadline for updates during the playing phase. This is because someone updating their levels during the creation phase isn't really an indication that the creation phase is still going; as this could just as easily be done during the update phase (which would extend as long as the authors feel further updates need to be made).

One point also came up - technically, there's no rule stopping someone from submitting a blank or incomplete level during the creation phase, then finishing the level off during the update phase. Obviously, this isn't the intention of how the update phase should be used, but it would technically be allowed. I figure that anyone who uses this kind of loophole in the first place (which I'd think unlikely in this community) would be equally likely to try and loophole any rule that tries to prohibit it, but perhaps a better solution is to allow for someone to simply indicate their intent to submit a level, and allow them up to the end of the update phase to actually submit one (perhaps with a rule that, if they indicate intent but do not submit a level in the end, they're prevented from simply indicating intent for the next few contests (but could still enter if they actually submit a full-fledged level during the creation phase; and it is of course possible that such a rule could be made to only apply to repeat offenders)). One of the reasons why I see some kind of system like this as nessecary is due to the contests only going ahead if 3+ entries are received (and a difference in the number of prizes between whether 3 entries or 4+ entries are received). I feel that if it's simply a matter of "anyone can join in arbitrarily until the update phase ends", then there's a significant risk of people not submitting their levels by the end of the creation phase, which would skew the number of contests that actually go ahead - hence why I prefer a system of either "levels must be submitted before then" or at least "intent to submit must be indicated before then".
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on June 18, 2015, 02:26:47 PM
It seems that so far, there isn't really any consensus on what to do about the deadlines. Of course it's still open for input, but assuming no general consensus is reached before the next contest, I'm going to go with my preference, which is to extend the initial submission deadline only if new submissions (not updates to existing ones) are received, and keep the status quo for the update deadline.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: Akseli on July 08, 2015, 04:09:57 PM
I've been missing full score breakdowns in these contests. So far only placements have been revealed, and only for the levels that have made it to the final round. Vote results for each round have always left unrevealed, even though at least I would be very interested in seeing those. Here is an example of thorough results. (http://www.lemmingsforums.net/index.php?topic=1571.msg40362#msg40362) The results should be revealed after the whole contest is over, and not in parts after every round, for example. This is pretty much the same issue I hoped to see in the voteoffs for the official games, now we don't know how levels succeeded between each other in their own rounds.

So, if no one finds revealing full results a bad idea, I would like to see this in the contests.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on July 20, 2015, 02:34:57 AM
In regards to SuperLemmini - it seems that this engine is very unpopular. Very little content has been released for it, as far as I can recall it's never been used in a contest entry, and development seems to have halted on it now.

With this in mind, I am considering removing SuperLemmini from the list of accepted engines in future contests (regular Lemmini will remain accepted). Any thoughts on this?
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: Crane on July 20, 2015, 06:30:41 AM
It's a shame because I consider SuperLemmini to have some potential, but with just some annoying quirks that might need to be ironed out, or otherwise some functionality that clashes slightly with other versions of Lemmings.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on August 15, 2015, 06:05:01 AM
It'd appear I spoke too soon about development being halted; but it does remain an unpopular engine for anything beyond playing the official levels. For now, I won't make any changes to its status of being an acceptable engine; although it appears the contest has now settled into a setup of "majority are NeoLemmix, with a handful of Lix levels, and other engines aren't being used".


Anyway; a point I'd like to discuss is naming for updates. We've had cases where pre-playing phase updates have been labelled "V2" or "V1.1"; and one case where someone labelled their update "V2b". I've ignored these in maintaining the list, instead naming the files (and referring in the topic's first post) as simply "V1", "V2" etc; with the state of the level at the beginning of the playing phase as always "V1", even if there's been updates during the creation phase.

I'm wondering if the rules should explicitly specify a guideline on how update versions should be named, to minimize confusion that may arise from such...
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on April 26, 2016, 06:18:41 PM
So, the latest contest (contest #9) attracted a very large number of entries. Given that; I'm wondering if I should use this choose-one-of-three-rules format more often - and if so, should I make it the standard format, or just a once-in-a-while thing? As expected, I put a poll up. :P

In chat, a point came up about how this could result in too many level entries. One suggested precaution against this is giving three options, but only allowing each user to enter up to two of them. The possibility was also discussed of simply doing it as two-options rather than three-options; but I prefer the three-option-max-two structure for one simple reason - some ideas might only be possible in certain engines (such as in contest #9, the first idea was only possible for NeoLemmix or Lix), so rather than having to avoid those kind of rules to avoid reducing entries, I can simply ensure that every engine has at least two rules that are applicable to it.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on April 27, 2016, 05:50:30 PM
It seems there's a lot of approval for the idea of making the three-rule structure a permanent thing.

As mentioned in edits to the previous post, the most likely setup will be that three rules are given; of these, each user may enter up to two of them. In any given contest, every allowed engine should have at least two rules that can be made on it (this is to ensure everyone can enter two levels if they want regardless of engine preference; while not precluding the use of rules that are dependant on (most likely) NeoLemmix or Lix to work).

It was discussed in chat how feasible it may be to keep making new rules for this, especially given that the rules in a three-rule setup will generally need to be more general than those which could apply to a single-rule contest. Most people felt that it was okay to recycle old rules to some extent, as long as there were new ones being used too; one user proposed specifically using a guideline of "two new rules + one old rule each contest".

Note that this doesn't mean every contest will be a three-rule one; just that most will be. From time to time, there may be exceptions, the most obvious example being future Level Of The Year contests, as well as tag team contests, or just a one-ruler once in a while to mix things up a bit.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on May 30, 2016, 04:42:34 PM
In the #neolemmix IRC channel, there's been some people who question the point of the deadline extension rule.

The initial intent behind this rule is so that entries / updates don't close until no more entries / updates are coming in. In this latest contest, we've seen the deadline extend multiple times, almost at the last minute, for the entry period. This could indeed be quite annoying to those who just want to get to the stage of playing the levels.

I'd like to give everyone a chance to offer their input - should this be changed? Any changes won't affect the current contest (as usual), but will apply from the next contest.

The main ideas I can see being worthwhile are (but, feel free to suggest others!):

1. Keeping it as it is now
2. Putting a limit on the extensions. This could be in the form of "it can't extend by more than X days from the original deadline", or in the form of "it can only extend X times, regardless of how long those extensions are".
3. Removing it altogether.

It is also very possible that one of these could be used for the initial submissions, and another for the update period. For example, we may want to suggest that in the initial submissions period, the deadline is set in stone; while in respect to the update period, the extension rule stays the same as it currently is.

Any input on this?

(On a similar note, currently, the playing phase continues for about 4 days beyond the end of the update phase. How do people feel about this - is it too short? Too long? Just right?)
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: Simon on May 30, 2016, 04:54:39 PM
No extensions, simpler rules. I can then plan ahead what weekend to meet with Icho.

-- Simon
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: Colorful Arty on May 30, 2016, 04:56:22 PM
I think we can remove the extensions
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: Flopsy on May 30, 2016, 04:57:08 PM
I think for the submissions stage only, we should definitely change the rule from what it currently is. Either have a deadline set in stone and it won't change (option 1) which would be preferable for me but I'm willing to accept option 2 as well because at least it appeases both sides of the argument to some extent.

For the updating/playing phase, the rule should stay the way it is. I think level editing/updates are very important in contests and we should not put a time constraint on that at all.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: Wafflem on May 30, 2016, 05:08:05 PM
Just remove the extension from the initial submission stage.

Also, the forums have been having a lack of activity these past few days (in fact, these happen frequently). Having a playing phase is a great way to put some activity in the forums again.

For the extension to the update phase, since we already have so many weeks of the playing phase in the last contest, the extension feels pointless to me. But it's up to you.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: Proxima on May 30, 2016, 05:09:00 PM
I feel rather mixed about this, since I am usually very busy and find it hard to find time to make contest entries -- it's always nice when there's a deadline extension so I have a bit more freedom to plan which day I will work on my levels. The flipside is that contest entries have to be a fairly low priority, so I will usually submit close to the deadline, which results in a further extension and annoyance for others -- though at least in this case, one person was able to submit in the extended deadline when he might have missed out. But let's be honest, if there were no deadline extensions and it meant I didn't get to submit, that's fair enough, it just means that other things were more important at the time, and I'll get the chance to submit to other contests. Now that we have the "3 rules, 1 old rule" system I don't have to worry about permanently missing out on an interesting rule, either.

I think the fairest system would be to go back to the rules of the contest geoo ran before namida's official contests -- no deadline extension, but you can still submit a level during the playing-and-updates phase, there is just some penalty for doing so. Not sure what would be an appropriate penalty under namida's voting system. Maybe deduct 1 from the level's vote total in the first round?
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on May 30, 2016, 05:41:26 PM
Quote
I think the fairest system would be to go back to the rules of the contest geoo ran before namida's official contests -- no deadline extension, but you can still submit a level during the playing-and-updates phase, there is just some penalty for doing so. Not sure what would be an appropriate penalty under namida's voting system. Maybe deduct 1 from the level's vote total in the first round?

This is also a tough one. I don't feel penalizing a level itself just becuase it was submitted during the playing phase is fair, if such submissions are allowed in general. However, I'd want to encourage as many people as possible to submit during the initial creation phase. I'm open to the idea in general, though. Maybe one possible answer, at least in the case of three-rule contests, would be to not allow users to submit a second level during the playing phase - so the level itself is not at any disadvantage, but the creator is only able to enter one level unless they enter both levels during the initial creation phase?

Alternatively, perhaps the phases could simply be merged altogether; so as soon as a level is submitted, it's available to play.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: Dullstar on May 30, 2016, 10:31:21 PM
I'd suggest only extending the deadlines if there's a shortage of levels and there's reason to believe an extension would result in more entries.  Otherwise, it's probably best to stick with the original deadline.

If finding time to create levels is an issue, it may be worth considering increasing the length of the submission phase but keeping the deadline consistent rather than a shorter period that can be extended indefinitely.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on August 01, 2016, 04:29:46 PM
For the new contest, I've made the following changes (I also mentioned these in the contest's topic itself):

- The initial submission phase only extends if there aren't many entries yet. Specifically, it no longer extends for entries after the 6th one.
- New submissions are allowed during the update phase. At this point in time, there's no penalty for doing so.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on August 24, 2016, 03:59:44 PM
A proposal came up in chat the other day of whether the less-popular engines should be removed from the accepted engines list. This would essentially mean that future contests are restricted to only NeoLemmix or Lix. (Although Lix is noticably less popular than NeoLemmix for contest entries, it is definitely popular enough to keep it allowed.)

How do people feel about this? On one hand, it means that while playing the levels, we won't have cases where people get frustrated at the relative lack of fine control features in other engines. However, a lot of new users have only switched to these engines after initially submitting levels in other engines (most commonly SuperLemmini) and then properly discovering NeoLemmix or Lix through the contests, so in this way, it might be harmful overall to make this change.

On a semi-related note, we also had a situation in this latest contest where a user provided both NeoLemmix and SuperLemmini versions of their level. In this case, the user later decided to withdraw the SuperLemmini version and simply keep the NeoLemmix version in the contest, although there was no rule specifically obliging them to do so (indeed, there's no rule that would cover this situation at all, unless we were to interpret it as being two seperate entries that just so happen to be identical except for what engine they run on). I most certianly do not want to require that all levels be cross-engine, especially with the difficulties in converting a NeoLemmix or Lix level to any other engine (converting a Lemmix, Lemmini or SuperLemmini level is usually not so difficult). Do we want to allow this (and if so, do we want to require the entrant to nominate a "primary" version), or should we add a rule against this? If nothing else, I'm leaning towards disallowing it for the sake of keeping things simple - especially if we do also restrict future contests to NeoLemmix and Lix only.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: Colorful Arty on August 27, 2016, 12:07:28 AM
I know I am in the minority here, but I personally still like SuperLemmini, but I am willing to switch to NeoLemmix only. I would prefer to allow other entries as well, but unless said levels require specific tricks not present in NeoLemmix, then it does make more sense to convert them to the preferred format.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on August 27, 2016, 02:36:21 AM
I put up a poll, so those who want to state their opinion without writing up an explanation as to why (or revealing who feels that way) can give their view.

I added an intermediate option of only allowing the modern engines - effectively, just removing Lemmix and Lemmini - which it should be noted that these haven't been used for the contests in a long time anyway.
Title: Re: Official Level Design Contests - Discussions, rule debates, etc
Post by: namida on October 06, 2016, 04:04:16 PM
It seems opinions are fairly evenly spread, with a slight bias towards reducing the options a bit.

Unless further voting / discussion suggests otherwise, from contest #12 I'll probably remove Lemmix and Lemmini from the allowed engines list (it's been ages since we've had an entry on either of them anyway), but leave SuperLemmini for now (although small in number and often not very successful, we do still get SuperLemmini entries from time to time).