[SUG] Visual SFX

[jkapp76]

I would love to see the graphic sound cues from lemmings 95. This would allow deaf people to know when the builder was laying his last 3 bricks. (and they're really cool to see)

These images are included in the 95 version. And the (super lemmini too) fork has these and even created a couple extras I believe.

[WillLem]

Yeah, this is something I can look at adding, for sure.

What do people think of the idea of having more picture/animation-based sound cues (rather than words), in order to further optimise the accessibility potential? Here's an example:

[Simon]

Very good, and all our games should do it.

-- Simon

[jkapp76]

I like the idea of adding more visual cues to complete the accessability features. But I really like the original text cues too. I think the best option would be to use the originals but create some new ones like you show here.

A good cue for when a distant lemming falls to his death would be nice...

[mobius]

Yeah, this is something I can look at adding, for sure.

What do people think of the idea of having more picture/animation-based sound cues (rather than words), in order to further optimise the accessibility potential? Here's an example:

I like that idea, over the words in win95, which I always found a bit silly-looking.

[jkapp76]

I like the look of these non-text cues too. But the Yippie, and No! and maybe others seem to require text. Maybe not?

Here's all the cue images...
Here's also a version of the above pic with the text label... "Chink". The text version is much more visible.

[WillLem]

Just giving this a bump. After the next release (likely later today or sometime this weekend), I'll start to look at how to draw images to the play screen.



Note to self: As well as the visual SFX, I'd also like to draw a clock over the exit when time runs out, and display a centre-screen graphic warning the player when only zombies remain and/or when no lemmings remain but the level isn't completed (both of the latter will also require the level not to exit in these circumstances.)

[WillLem]

Wow, so... drawing these to the screen is not easy.

It's a very one-step-forward, two-steps-back kind of process. At one point I did manage to draw one of the graphics to the gameplay area, but the timing was way off. And now, something has changed in the code since I last looked at it, so I can't even draw it at all. I'll probably figure it out after a good night's sleep and a cuppa.

I dislike the WinLemm text tbh, I always turned it off when playing on that particular engine. But, I can see how it would be useful when playing without volume, and for accessibility reasons. I'd like to either redesign these, or else explore the idea of picture/symbol-based cues more fully to see what we can come up with.

I'll probably need help with this one, tbh. We want to do this in the most optimal way possible - anything graphics-related tends to guzzle resources. I'll get in touch with EricLang and see if he has any suggestions for how to get started and how to minimise framerate hits. However, if anyone else wants to help me out, please PM me and we can look at it together. I'll probably need an extra pair of eyes and a better brain than mine to take this one on!

[WillLem]

Had another look at this today and happy to report that I've made some limited but significant progress with it. I can now draw stuff to the screen at particular times and for a particular duration - the recent update to the Freezer skill has helped with figuring this out.

However, since there are multiple images, times and drawing-locations to manage, it's still a considerably large task which I'm beginning to think may even require its own unit.

The end goal is to have a neat-looking set of Visual SFX for different in-game situations (beginning, exiting, running out of bricks, splatting, falling off-screen, exploding, etc) and have them display in easy-to-see but not intrusive locations.

They will be optional, of course, and more than that - I'm thinking it might even be a good idea to have the option to either display the graphics at relevant in-game locations, or always centre-screen. The latter may come in more useful when using the feature for accessibility reasons, or when the sound is off, because some actions may be happening off-screen and therefore may not be visible or audible.

[jkapp76]

I don't know how difficult this would be, but I think the best method would be for the "Tink!" to appear at the far left if the builder is
left and off the screen. And far right if he's off to the right. This would take care of the location for most of these. Anything else would
appear above the affected skill. Same with "Let's go!" It could be centered above the hatch. They might all be good to
be slightly above whatever is making the noise.

Center-screen seems okay too and should be easier to implement. Could start here and see how it goes.

P.S. In the screenshot above, the text appears to begin perfectly above the lemming and looks great. I think
off-screen sounds would look better at the edge though.

[WillLem]

the best method would be for the "Tink!" to appear at the far left if the builder is left and off the screen. And far right if he's off to the right.

Agreed, this does seem like the best idea.

I'd probably need to look at the code that gets the lem's position. Even then, I'm not sure if this knows whether or not the lem is currently in the screen area. Maybe there's already a way to determine this. If not, this could be very tricky.

As for timing, I'm currently using a global timer for these graphics, so they often cue when they're not actually supposed to. Even giving each graphic its own timer wouldn't necessarily work - for instance, if there are two Builders at the same time. I need better checks for whether or not to display the graphic, and maybe a better way to handle the timing.

Slow progress, this one.

[Herby]

Hello! The visual effects is nice idea!

Ideas for effects.

Above trapdoor.

Trapdoor is open - The door is open.

Above exit.

Lemming go to Exit - The sign with inscription ''Exit'' is the inscription on it flashes.
Time is up! - The clock minute arrow is crazy rotate clockwise.
There are not enough remaining lemmings to complete the level - Exclamation mark.

Above lemming.

Skill is gived. - Dot.
Oh-no! - The fuse on the bomb burns out.
POP! - The bomb explode.
Glub. - The bubble bursts.
Fry. - Lemming are burned and b**od is pours out.

Extra.

Lemmings on levels are zero, without minimum saved. (without zombies) - The big warning sign on middle of screen.

[WillLem]

Some nice ideas there Herby, thanks for sharing these

To begin with, I suggest that perhaps we start by finding a way to visually represent the "chink" sound for Builders et al, and also look at how we might be able to show off-screen deaths visually.

At this point, Herby's idea of using symbols such exclamation marks, dots and clocks rather than words seems like a better idea than using the WinLemm text graphics (which are a bit clunky-looking IMHO). I also still think the idea of a simple 3 lines for Builder bricks works nicely, and this could potentially be added to the existing Builder animation sprite rather than displayed as a separately-rendered graphic:



These lines could then be displayed at the side of the screen, maybe slightly larger for increased visibility, when the Builder is off-screen. And similarly, maybe we can use Herby's exclamation mark idea for off-screen deaths:



If I can figure out a way for the game to know whether or not the action is happening outside of the currently-visible screen area (by no means trivial, if even possible), then all that's needed is to display the relevant graphic at the side of the screen (also not trivial, but definitely possible) - this would then sufficiently alert players that something is happening that needs attention, and whereabouts it's happening.

Technical concerns aside, this feature definitely seems worthwhile and clearly has a decent amount of support. The topic will remain open and I'm more than happy to continue discussing ideas, but it might be a while before we see it. It feels important to get this one right.

[WillLem]

What do people think of this as a possbility for displaying "something is happening elsewhere" Visual SFX? Note that this is very much a mockup; the graphics would most likely be improved upon, probably made to look similar to the panel buttons/minimap border - this could be decided upon later.



I'm still not sure if this is even possible (i.e. the engine knowing that something is happening outside of the visual area of the screen). Also, these would also need to always appear the same size regardless of zoom factor. I imagine that this is possible due to the fact that the panel can zoom separately from the gameplay area, but it would probably need entirely its own rendering logic.

[jkapp76]

I like it a lot. I would have used yellow and red in place of grey and green... but I like the look.

[Tygerboi]

I am liking the ideas put forward here by Herby and others.  I did find the "chink" vfx very useful in the past when I have played WinLemm and SLT with the sound off - for example, in the living room whilst the children are watching TV, if there's also a conversation going on in the room, but could also maybe be if SLX was played whilst in a public location such as on public transport, a hospital waiting room, etc.


I didn't think the "chink" text looked too bad.  Maybe if text vfx were implemented, there could be an option to either have them turned off (default), turned on on their own, or turned on with symbol vfx, although the last option may not work very well (having both activated) unless the symbol vfx that had equivalent text vfx were disabled if both were activated.  That sounds like it  would be very fiddly though, so options for either text, or symbol or none, would be the best solution?


That idea for off-screen vfx to the left and right of the screen is fab!  Could be very useful for sound-off/hard-of-hearing situations,  The only thing that could be an issue is the vfx obscuring what is occurring on the viewable screen area.  For instance, if a builder was building a bridge near the edge of the screen whilst another builder was nearing the end of the 12-brick run off the screen on the same side, and there was also other on-screen activity going on that produced vfx, it could all start to get too cluttered/confusing.  One way around that might be to have different colour vfx for off-screen and on-screen (as possibly implied by one of the screen mock-ups).
It might be pretty cool too if, for the builder vfx, if three bars would appear when three bricks remain, two bars, when two, one when one, then the exclamation mark when the builder is about to give up and walk on.
Is there really any point in a "Let's Go" vfx?  It's already visually indicated when lemmings start to fall out of the entries because you can see lemmings falling out of them   Still, if it's optional and not too difficult to implement alongside other vfx, then why not, I guess.


The timing issue for off-screen effects seemed to work just fine in WinLemm and SuperLemminiToo, although in both cases they are only audio not visual.  Does having the visual element as well make the timing process much more difficult, then? 

[WillLem]

I didn't think the "chink" text looked too bad.  Maybe if text vfx were implemented, there could be an option to either have them turned off (default), turned on on their own, or turned on with symbol vfx

Text, symbols or none could work, this is a good idea. Of course, it would always be possible to swap out the graphics for whatever you like anyway, but maybe this is one to support at base level for accessibility reasons. It should only be a small handful of files anyway.

That idea for off-screen vfx to the left and right of the screen is fab!  Could be very useful for sound-off/hard-of-hearing situations,  The only thing that could be an issue is the vfx obscuring what is occurring on the viewable screen area.

Definitely worth considering, particularly if it ends up being very difficult (or, not at all possible) for the game to "know" whether something is happening "off-screen", or to draw graphics to an overlaid layer (which doesn't currently exist, and would eat up rendering resources).

Perhaps the hints could be displayed at the top of the game window, in the centre, with an arrow pointing left and right, something like this:



Or, more preferable, we could even use the panel display (this is coming in handy for a lot of things recently!):



Obviously, this would temporarily obscure the display, but the likelihood is that whatever is happening to prompts the hint is more important anyway, and we can always make them clickable-to-stop-showing.

In fact, one very large benefit to using the panel is that we wouldn't need another rendering layer; this might end up being the way to go with it.

It might be pretty cool too if, for the builder vfx, if three bars would appear when three bricks remain, two bars, when two, one when one, then the exclamation mark when the builder is about to give up and walk on.

I like the idea, but let's keep it simple to begin with (i.e. the same graphic each time) and see how necessary this is.

Is there really any point in a "Let's Go" vfx? ...  Still, if it's optional and not too difficult to implement alongside other vfx, then why not, I guess.

The only two that I would consider necessary from an accessibility point of view are the Builder/Stacker/Platformer alert, and a lemming death that might have happened off-screen. All others are purely cosmetic. At some point I'll get a list together of the various sound cues that might need Visual SFX and we can decide what to do about each.

Does having the visual element as well make the timing process much more difficult, then?

Not really, it's essentially the same process of passing a message per-lemming to cue the graphic. I understand a bit better now how things work than when I first attempted to implement this, so my next attempt should be more successful.

In fact, in order to bring this feature closer to something that feels manageable, I'll go ahead and make the call that off-screen hints will be shown in the panel area. This will greatly simplify the feature; we already have code to get the Lemming's position, and we already have code to draw a message to the panel.

[jkapp76]

I think the hearing impaired would appreciate a verbose mode that includes Let's go and Yippee as well. If it's all going in the panel anyway.

I prefer a nice bright contrast myself, like a yellow or bright orange for the text.

Will the text just stay for a couple seconds and then disappear?

[WillLem]

If it's all going in the panel anyway.

I'd suggest that only the "off-screen and vital" stuff should go in the panel display - everything else can simply be displayed on-screen adjacent to the relevant lemming/item as it is in WinLemm/SLToo.

I prefer a nice bright contrast myself, like a yellow or bright orange for the text.

I dislike the WinLemm ones tbh, but I'm happy to use the same colour scheme and design some new ones. You can always swap the graphics for the WinLemm ones if preferred.

Will the text just stay for a couple seconds and then disappear?

Yes. And, if in the panel, it will be clickable-to-remove.

[WillLem]

Well, I love a good spreadsheet so here's what we can possibly do for each of the sound cues :

Updated table is here.

[Tygerboi]

Sorry for replying this way, but I've been having problems replying to WillLem's reply on 26/04/2024 with it shrinking the text size of my responses down to about size 1, so is only readable at 500x zoom!  Also, the preview button/Alt-P doesn't actually show me a preview.
Anyway, on with my reply:

I'd suggest that only the "off-screen and vital" stuff should go in the panel display - everything else can simply be displayed on-screen adjacent to the relevant lemming/item as it is in WinLemm/SLToo.

I agree with that, as the on-screen ones will be harder to miss, and it would be helpful having them over the relevant lemming.

Will the text just stay for a couple seconds and then disappear?

Yes. And, if in the panel, it will be clickable-to-remove.

Why click-to-remove the panel display messages?  Wouldn't this potentially cause confusion if, say, a builder off-screen starts to run out of bricks so a message is generated but then stays there until the message is clicked, but you also have initiated another builder or two elsewhere that may happen to be off-screen, and you have the sound off, how will you be able to tell whether the visual prompt in the panel is an old one that you forgot to click to clear, or a current one that needs immediate action?  Also, would there ben enough panel space for multiple messages with directional indicators to be displayed simultaneously?  All of these off-screen activities that need prompts are all time-sensitive in that you have to either let the lemming go or else reassign a task within 3-5 seconds, so the warning prompts might as well also only be displayed for 3-5 seconds, like their on-screen counterparts.  Having to remember to click off messages during the heat of the moment is also a "passive" (i.e. doesn't directly affect the gameplay) player task that could result in missing the timing for assigning a lemming skill at exactly the right moment, or something along those lines, unless the pause button was extensively used.  Which reminds me - if you have a visual prompt on-screen over a lemming (e.g. "chink!") and the pause button is pressed whilst its on-screen, will that also stop the message disappearing until the pause is released, or will it still time-out and disappear?  I'd prefer the first option - pause game also pauses on-screen/panel message time-out countdown - as it could potentially help with planning how to complete a level.  It may also be useful for taking screenshots.

[WillLem]

I've been having problems replying

Yes, I noticed this. I've taken the liberty of editing the affected posts for readability for now, but definitely post about it in Site Discussion if it continues (it seems to only be a recent thing for you?).

Also, the preview button/Alt-P doesn't actually show me a preview.

Again, probably something to mention in Site Discussion if you're having regular issues with it. In particular, we could do with knowing what computer/phone/etc & browser you're using to navigate the Forums, as it may be a browser-specific issue.

Why click-to-remove the panel display messages? Wouldn't this potentially cause confusion ... how will you be able to tell whether the visual prompt in the panel is an old one that you forgot to click to clear, or a current one that needs immediate action?

I should probably have been clearer about that. I meant the messages could be clicked to remove them optionally, for example if the player wants to be able to see athlete info, or any other info that uses this area of the panel's display.

By default, the message will probably be displayed for 3-5 seconds or so, or until the message is no longer relevant, whichever is soonest.

Also, would there be enough panel space for multiple messages with directional indicators to be displayed simultaneously?

Probably not tbh. This is where we hope that such limitations promote a creative solution!

For example, in a case where multiple Builders need attention at the same time, we can either simply continue displaying the same message, or display a custom message ("Multiple Builders are running out of bricks!"). Either could potentially work.

if you have a visual prompt on-screen over a lemming (e.g. "chink!") and the pause button is pressed whilst its on-screen, will that also stop the message disappearing until the pause is released, or will it still time-out and disappear?

We'll almost certainly pause all Visual SFX along with everything else. Hopefully, the rendering will already handle this by default, but I'll know what to do about it if it doesn't anyway.

Thanks for the queries. Keep these coming: questions and comments help the feature take shape

[WillLem]

Well, I honestly thought this one would be a step too difficult, but thankfully I've managed to find a way to determine whether or not a lemming is within the viewport, and along which edge (top, bottom, left, right) - see commit ff116ed if you're interested.

What this means is that, however Visual SFX ends up being implemented, we can definitely factor in whether or not an event is happening offscreen, and importantly exactly where it's happening.

I think that, to begin with, I'd like to keep this feature as simple as possible and use as much of the existing UI as possible. Adding another rendering layer isn't a tempting prospect, especially since SLX is only just about managing to deal with the existing rendering load.


So, here's how I think the Visual SFX feature could look, based on what we have:

Sound Cue	Visual SFX	Panel Message (off-screen events only)
Level Start	"LET'S GO!" (above entrances/pre-placed lems)	N/A (off-screen level start areas shouldn't be formally supported)
Zombie Level Start	"BRAAAINS!" (above zombie entrances/pre-placed zombies)	N/A
Builder/Stacker/Platformer Warning	"CLINK" (above working lems)	<- Bricks are running out! ->
(Time)Bomber 'Oh No' State	"OH-NO!" (above lem)	N/A
(Time)Bomber/Freezer Explosion / Balloon Pop	N/A (seems unnecessary, since these already have plenty of visual cues)	N/A
Hit Steel/OWW	"BONK" (above lem)	<- Lemming hit steel/OWW ->
Lemming Death	"YIKES!" (above lem)	<- Lemming lost ->
Time Up	"TIME UP" (above exits)	Time is up!
All Collectibles Gathered	"WELL DONE" (above lem)	N/A
Button Click	"CLICK" (above lem)	<- Button Pressed ->
Exit Unlocked	TBA	<- Exit Unlocked ->
Lemming Exit	"YIPPEE!"	N/A

The main questions are:

• How long should we show the panel message for?

Currently, I'm thinking they should appear until the relevant lemming/event to which they are referring is brought into view. They should also be click-to-cancel (optionally), so the player has a way to acknowledge the message and remove it from view.

• Which GFX should we use?

The Superlemmini/WinLemm ones are OK, but I think we can do better. I'll have a look at doing something myself, but I'm very much open to ideas and suggestions on this. If you want to get involved and provide some graphics, I'll definitely at least give them a try out.

Let's try to keep each one to no more than about 48 x 48 in hi-res and 24 x 24 in low-res. That's obviously a rough guide; if they're bigger than that and look great, then of course they'll be a decent candidate.

• Should we keep them hi-res only?

One of my goals with SLX is to eventually make everything but the game itself hi-res (cursors, helpers, etc); the panel was a great start and works nicely, but anything that is displayed in-game is necessarily problematic due to relative size/positioning. I think it could be do-able though: currently, everything is re-sized when zooming in, so - in theory - it should be possible to increase the minimum zoom to 2 in low-res, and display the images at their original size at that zoom factor, scaling up from there. In theory.

It's probably best for me to have a go at that first before we make any decisions for the Visual SFX. I'll keep this topic posted.

Suggestions/Thoughts welcome.

[GigaLem]

Maybe rename the ticking sound of the builders to "CLINK" as while I do respect the original sound effect or onomatopoeia for legacy as I know there's no ill intent with it, however the word is a bit poorly aged as the word "Chink" unfortunately is also an offensive slur, so much a restaurant had to be renamed from it.

I don't like bringing up the subject and such because I feel it might come off as unnecessary, but I tend to be cautious about things.

[WillLem]

Maybe rename the ticking sound of the builders to "CLINK"

Good shout, let's bring it up to date. I've updated the table accordingly and if we use a word I'll use "CLINK" or maybe something else. I'd probably go with a symbol rather than a word wherever possible anyway.

EDIT: I've also updated the name of the sound itself, along with renaming a number of other default sounds to names which make more sense (e.g. 'assignskill' instead of 'mousepre', 'brick' instead of 'ting', 'boing' intead of 'oing').

As a side-note, it's now possible (as of the next update (2.8.5)) to replace default sounds per-theme and per-levelpack.