Author Topic: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates  (Read 6457 times)

0 Members and 1 Guest are viewing this topic.

Offline namida

  • Administrator
  • Posts: 12398
    • View Profile
    • NeoLemmix Website
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #15 on: October 30, 2019, 06:22:26 PM »
Quote
The alternative option may be to go back to how we used to do things - NL comes only with a few core styles, and users must download any extras on their own. Content authors would need to point out what styles are used in their packs, and ideally, where to get them. Downside of this, is that if someone's download link breaks, the style is lost, and all content that relied on it is broken. Also, we now have a whole bunch of content that uses a mix of "core" and "non-core" styles, and creators of this would need to go back and figure out what's included vs what needs to be downloaded.

I am 100% against this. I would rather download 1000 styles than tracking everything down manually and deal with tons of broken links and new topics about missing styles errors like in the old days (Seriously there was one every week or so).

I see that we need a bit more quality control though to not let the number explode unnessesarily. Maybe for any future submissions exclude slight recolors or near duplicates to avoid any further excessive duplication.

For the older near duplicates: Exactly list them, examine the usage and then decide over a possible exclusion.

One thing that I was reminded of while thinking about a different matter: This situation (with missing styles) only arose because of content creators ignoring the repeatedly-given advice, "your NXPs should include a copy of ALL non-standard styles used in your packs". Of course, this is not practical in new-formats where styles included with packs will start overwriting each other (possibly with out of date versions), but yeah, issues back then occurred primarily due to blatant disregard for best practices.

Another piece of advice I have repeatedly given, relating to the current formats, is that packs should be ZIPped such that they can be extracted to the base NL folder (ie: the ZIP will contain a "levels" folder, inside which is a "InsertPackNameHere" folder; it might also contain at the base level a "music" or "styles" folder if needed). This has been even more severely ignored; I don't think I can name a single pack outside of my own that is ZIPped in such a manner, and I've pretty much given up on trying to get this message through. The NL installer needed signifcant kludges in its coding to deal with this. At least most packs are zipped in such a way that you at least only have to extract it once; but I've seen a few where you have to "extract these files to levels, then extract these ones to music, etc"... I didn't even try to make the installer account for this; instead either rejecting or re-packing affected packs (depending on their value).

The point here: Any solution must be enforced somewhere along the line - be it in NL itself, or a server that content is uploaded to. It's not going to be good enough to merely advise people "do this to avoid problems", because no one will do it. Even things that must be done to avoid content breaking are often semi-neglected by some (but not all) users.
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline IchoTolot

  • Global Moderator
  • Posts: 3608
    • View Profile
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #16 on: October 30, 2019, 06:57:29 PM »
Quote
Another piece of advice I have repeatedly given, relating to the current formats, is that packs should be ZIPped such that they can be extracted to the base NL folder (ie: the ZIP will contain a "levels" folder, inside which is a "InsertPackNameHere" folder; it might also contain at the base level a "music" or "styles" folder if needed). This has been even more severely ignored; I don't think I can name a single pack outside of my own that is ZIPped in such a manner, and I've pretty much given up on trying to get this message through. The NL installer needed signifcant kludges in its coding to deal with this. At least most packs are zipped in such a way that you at least only have to extract it once; but I've seen a few where you have to "extract these files to levels, then extract these ones to music, etc"... I didn't even try to make the installer account for this; instead either rejecting or re-packing affected packs (depending on their value).

I didn't even know this was a thing to be honest.

Also, at least for me, I never ship the music together with the pack as then the download just to get an update becomes too large, therefore I have 2 components. And for these I personally always prefer unzipping inside the levels and music folder respectively as that seems more intuitive for me (pack into levels, tracks into music). I never even had the installer in mind and didn't know there needed to be bigger adjustments.

Quote
The point here: Any solution must be enforced somewhere along the line - be it in NL itself, or a server that content is uploaded to. It's not going to be good enough to merely advise people "do this to avoid problems", because no one will do it. Even things that must be done to avoid content breaking are often semi-neglected by some (but not all) users.

True words.

Offline namida

  • Administrator
  • Posts: 12398
    • View Profile
    • NeoLemmix Website
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #17 on: October 30, 2019, 07:15:46 PM »
Quote
Also, at least for me, I never ship the music together with the pack as then the download just to get an update becomes too large, therefore I have 2 components. And for these I personally always prefer unzipping inside the levels and music folder respectively as that seems more intuitive for me (pack into levels, tracks into music). I never even had the installer in mind and didn't know there needed to be bigger adjustments.

Seperate ZIPs is okay; it's about where those ZIPs should be extracted to - even before the installer, this was a thing simply to keep it simple - the user always just needs to extract to the NL folder, and not worry about what things go where in terms of subfolders.

Long term, this should become a non-issue user-side, as NL itself downloads and extracts the packs. But this is not a feature NL has yet; and at any rate this would make it more important to have consistency creator-side.
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline namida

  • Administrator
  • Posts: 12398
    • View Profile
    • NeoLemmix Website
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #18 on: October 31, 2019, 02:04:15 AM »
Flopsy made another topic asking about a specific case (duplicating single pieces across a few styles, rather than duplicating styles as a whole).

Most important post is the last one:

Had some uncertianty on Discord around the original question so just to be clear:

- Don't create styles that are, as a whole, slight visual variations on existing ones. I'm probably going to grandfather in existing styles of this nature (including, retroactively, orig_dirt_md), but we definitely won't be accepting new ones (at least for now; and this extends to the Genesis styles that weren't already present).
- It's fine to duplicate a few individual pieces between styles when they really do fit into every style - some official styles do this already (eg. steel blocks between orig_dirt and orig_pillar). For example, Flopsy's original question above would be a case where this is okay.

In terms of "what happens later if/when we get an ability like the one suggested in reply #2" - to be on the safe side, when duplicating pieces between styles, try to keep the name the same in all styles. But I envision that "fixing" levels where this is relevant will be as simple as "run Cleanse Levels in a version of NL that's new enough to have this feature".

In terms of "what's going to happen long term" - I do really agree that once we get on-demand downloading of styles back in place, this is a much less significant issue. The on-demand downloading is a priority feature for V12.8, so once that's out, we can possibly go back to allowing this. Before that though, a discussion needs to be had around physics consistency - whether NL enforces it somehow, or if it's on a basis of "not keeping it accurate to the source style is grounds for your style being removed", or whether it's just "use these styles at your own risk, don't assume they're physics-accurate to the source material".

I'm going to close this topic; further discussion around styles acceptance / tidyup goes in the original topic on this subject: https://www.lemmingsforums.net/index.php?topic=4483.0
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline namida

  • Administrator
  • Posts: 12398
    • View Profile
    • NeoLemmix Website
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #19 on: October 31, 2019, 02:06:57 AM »
A question I have for those who care about these "palette swap" styles (I'm going to use this term from now on, even though it's not accurate on a technical level, because it carries the right implication):

Do you envision yourself using the original version of a style and the palette swapped version of a style, together in the same level*? Or would it be acceptable for a setup of "you can only pick one version or the other"?

* other than cases where this is only done because of a piece that exists in the original but not the palette-swap version
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline ccexplore

  • Posts: 5311
    • View Profile
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #20 on: October 31, 2019, 08:59:21 AM »
I mean, I can conceive of level visual designs that might intentionally use both together in the same level, like maybe one side of the level is where your normal lemmings enter and gathered in, and the other side is zombie-lemmings infested, and a suitably palette-swapped set of terrain pieces (and maybe even objects) used to depict the zombie side differently from the non-zombie side (eg. maybe grayscale for the zombie side).

But on the whole, I expect such mix usage to be relatively rare.

That said, perhaps the level file format could even be extended to support applying per-element palette swaps, meaning in extreme case every single individual piece of terrain and object in the level can each have their own independent palettes/recolors.  In practice going to such extreme direction should be rare and perhaps somewhat discouraged (I imagine it might increase memory consumption, especially for objects), but in theory one could consider implementations along those lines, and then it would certainly allow level designers to use same style with multiple palettes applied all within same level.

Even the extreme case I just mentioned might not actually be so crazy design-wise, though again probably not common.  So maybe not literally tweaking every single terrain and object pieces, but imagine a vertical scrolling level where visually, you want to depict the lighting being the brightest at the top and darkest at the bottom, as if the lemmings are descending deep down into a cave or the abyss starting from near the surface.  The cleanest way (but not actually achievable currently, I think) to do this is to apply a single continuous gradient of opacity, overlaid over the entire level area.  But an alternative approximation of same effect could be to recolor each terrain and object to be brighter/darker based on their vertical positioning.

Offline Strato Incendus

  • The King of Shimmiers (crowned by Flopsy ;D )
  • Posts: 1747
  • #RIP Spearer/Grenader (2020 - 2021)
    • View Profile
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #21 on: October 31, 2019, 09:53:21 AM »
I for one am pretty sure that I have in fact already used the original and the to-be palette-swapped version of a style on the same level. ;) So enabling this is definitely something I'd support!
My packs so far:
Lemmings World Tour (New & Old Formats), my music-themed flagship pack, 320 levels - Let's Played by Colorful Arty
Lemmings Open Air, my newest release and follow-up to World Tour, 120 levels
Paralems (Old Formats), a more flavour-driven one, 150 levels
Pit Lems (Old Formats), a more puzzly one, 100 levels - Let's Played by nin10doadict
Lemmicks, a pack for (very old) NeoLemmix 1.43 full of gimmicks, 170 levels

Offline namida

  • Administrator
  • Posts: 12398
    • View Profile
    • NeoLemmix Website
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #22 on: October 31, 2019, 06:04:15 PM »
Quote
That said, perhaps the level file format could even be extended to support applying per-element palette swaps, meaning in extreme case every single individual piece of terrain and object in the level can each have their own independent palettes/recolors.  In practice going to such extreme direction should be rare and perhaps somewhat discouraged (I imagine it might increase memory consumption, especially for objects), but in theory one could consider implementations along those lines, and then it would certainly allow level designers to use same style with multiple palettes applied all within same level.

I think it would be easier to just keep them as separate styles in this case.

Quote
The cleanest way (but not actually achievable currently, I think) to do this is to apply a single continuous gradient of opacity, overlaid over the entire level area.  But an alternative approximation of same effect could be to recolor each terrain and object to be brighter/darker based on their vertical positioning.

This is currently achievable, though it would need to be done with a large no-effect object, which would need to be specifically made to fit the level's height (it could be resizable for the horizontal axis). Of course, this would not affect the lemming sprites, due to them being drawn in front of all objects. Note that the object would need to specifically be "no effect" and not "background", as the latter gets drawn to the lowest layer no matter what.

I don't advise actually doing this. The impact on performance would be very significant, and the fact that it's a no-effect, not a background, also means that neither clear physics mode nor "disable backgrounds" option will hide it in any way.
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline ccexplore

  • Posts: 5311
    • View Profile
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #23 on: October 31, 2019, 06:44:56 PM »
and the fact that it's a no-effect, not a background, also means that neither clear physics mode nor "disable backgrounds" option will hide it in any way.

Wait, really? :o I'm surprised clear physics mode would still show no-effect objects, why would it if by definition the object would have no physics effects?

Offline namida

  • Administrator
  • Posts: 12398
    • View Profile
    • NeoLemmix Website
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #24 on: October 31, 2019, 10:21:10 PM »
Quote
Wait, really? :o I'm surprised clear physics mode would still show no-effect objects, why would it if by definition the object would have no physics effects?

Because objects that are legitimately no-effect have usually been marked as background objects. No-effect has generally been used for animating parts of objects that don't as a whole animate, such as exit tops; where it would be more confusing for part of an object (visually speaking) to suddenly disappear.

We have secondary animations now, but very few styles outside of the official ones have actually been updated to use them yet; most still use no-effect objects for this purpose.
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline namida

  • Administrator
  • Posts: 12398
    • View Profile
    • NeoLemmix Website
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #25 on: November 03, 2019, 01:45:39 AM »
As I have said I was likely going to do, I'm going to grandfather in any styles that would fall under this, but already exist in NL's styles download - but no new styles that are just slight tweaks of other styles will be accepted until further notice.

This also means that orig_dirt_md will be restored (but the other Genesis styles, as they did not already exist in NL's download, will not be made available as part of the standard styles download).
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline Dullstar

  • Posts: 2092
    • View Profile
    • Leafwing Studios Website (EXTREMELY OUTDATED)
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #26 on: November 05, 2019, 03:38:29 AM »
I'm not sure how likely it is that support for palette swap styles will be added, but during the discussion regarding high-res styles, I found myself wondering how high-res would interact with palette swaps that were designed with the low-res version of the style in mind, under the assumption that the high-res style may be likely to use more colors than the original style. I'd assume one of the two would have to take precedence, either using whatever the fallback for missing high-res graphics is, or skipping the palette swap process.

Offline namida

  • Administrator
  • Posts: 12398
    • View Profile
    • NeoLemmix Website
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #27 on: November 05, 2019, 05:33:05 PM »
Quote
I'm not sure how likely it is that support for palette swap styles will be added, but during the discussion regarding high-res styles, I found myself wondering how high-res would interact with palette swaps that were designed with the low-res version of the style in mind, under the assumption that the high-res style may be likely to use more colors than the original style. I'd assume one of the two would have to take precedence, either using whatever the fallback for missing high-res graphics is, or skipping the palette swap process.

People have shown interest in being able to mix different palette swaps of the same style together in a single level, so the simplest way to do this would be to keep having them as separate styles. If one has high-res and one doesn't, then upscaling will kick in for the one that doesn't. Essentially, NL itself will have no concept of a "palette swap", it's just another style.

This should be, as mentioned, fine once we have on-demand downloading instead of having to download and extract all the styles up-front.
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline namida

  • Administrator
  • Posts: 12398
    • View Profile
    • NeoLemmix Website
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #28 on: December 14, 2019, 10:45:34 PM »
https://www.youtube.com/watch?v=YG2OmQ1B96U

From V12.8.0 onwards:
- NL's main download will include a few styles: "default", "xmas", "special", "orig_sega", and all the L1 / ONML styles. (And even orig_sega I'm on the fence about. One could easily argue that if its inclusion is justified, so is the inclusion of the L2 / L3 styles.)
- Other styles can be downloaed either manually, or through NL. NL can do this automatically when encountering a level that needs the styles; and also has a menu to manually download any available style.
- An "all styles" download will continue to be provided. This download will be updated periodically; not necesserially every time a single style is updated.
- Additionally, "special" will be split off, so that it only contains pieces from official levels. All pieces in there belonging to other users will be moved into a "[username]_special" style, with the aliasing system used to prevent breakages of existing levels. (Note that this only applies to the game. Before the levels will be useable in the editor after the V12.8.0 update, they'll need to be run through Cleanse Levels.)

This mostly averts the problem with the styles download being huge. I need to give some thought to how sounds might be better handled in this system, but everything apart from that is pretty much good to go now.
« Last Edit: December 15, 2019, 02:02:30 AM by namida »
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)

Offline namida

  • Administrator
  • Posts: 12398
    • View Profile
    • NeoLemmix Website
Re: [DISC][PLAYER] Styles cleanup - Duplicates / near-duplicates
« Reply #29 on: February 15, 2020, 09:59:57 PM »
With V12.8.0's release and the return of online downloading, I no longer feel this is a serious problem. Just remember - if you're creating / maintaining such a style, it's your responsibility to keep it up to date with the normal version of the style, and it is not the normal version author's responsibility to tell you when / what they update; it's yours to pay attention to it.
My Lemmings projects
2D Lemmings: NeoLemmix (engine) | Lemmings Plus Series (level packs) | Doomsday Lemmings (level pack)
3D Lemmings: Loap (engine) | L3DEdit (level / graphics editor) | L3DUtils (replay / etc utility) | Lemmings Plus 3D (level pack)