For style designers: one possible solution could be to have a dev copy of the style that is not managed by the style manager, and a stable version that is. Unfortunately, this would be treated as a completely separate style by NL, which is a bit inconvenient, but fortunately swapping out the style is a modification that's easy to make using a find and replace operation as long as all the pieces have the same names.
Or use Git.
For custom pieces for existing styles: these should either be submitted to the style maintainer for addition, or kept in supplemental styles. Don't add non-standard pieces to existing styles, as it causes dependencies to become murky, as you wouldn't, for instance, be able to easily tell if a piece was modded in, or if it's part of the standard version of the style.
This is already the existing advice - that people should not add to the official styles, or other people's styles, unless it's done in coordination with the author. This is taken a step further in that if someone were to try and submit an unauthorized modification to someone else's (or an official) style, it would not be accepted into NL's styles downloads.
For maintaining the style manager: I imagine there's already guidelines in place about this, but in most cases I'd say styles with breaking changes should be outright rejected, essentially forcing the new version to be submitted as an entirely new style, with some way of marking the old style as deprecated (I believe it's possible to mark pieces as deprecated already, isn't it? But I'm not entirely sure what effect it has assuming I'm not wrong about it being possible).
Compatibility etc is the responsibility of the style's maintainer, ultimately. If it were to become a problem, I may reconsider this. I'll usually only consider rejecting updates / changes
to one's OWN styles if they've ignored a blatant "don't do this" direction (such as when one user tried to submit a bunch of new custom pickup skills during the time when I stated that the "don't do this" advice would be treated as an outright rule -
to be clear, this is no longer the case and custom pickup skills are no longer prohibited or even discouraged, as there is now a proper system in place to ensure they don't break every time a new skill is added); or if they had a lot of existing stuff that was broken due to failure to keep it updated, they had not chosen to withdraw it, but at the same time
were submitting a lot of entirely new stuff.
NeoLemmix has two relevant features here. The first is the "alias" system - this allows a style to specify that certain pieces, or entire styles, should be remapped to others. For example, namida_circuit had two pieces that were exact duplicates of each other since the very early days - one was meant to be horizontally flipped, but due to an oversight was not. To avoid breaking existing content both were kept for ages; but the alias system meant that the duplicate could be deleted and instead remapped towards the other one. The editor does not currently support the alias feature; but if a level is run through Cleanse Levels, the output will have the remap hardcoded into it (and thus be possible to open).
The other is the "deprecated" flag. Pieces with this flag will, by default, not show up in the editor (though there is a "Show deprecated pieces" option under the View menu). Aside from this, they work as normal - and in particular the flag has no effect in-game.