Author Topic: [FEAT] Rewind button / hotkey  (Read 1310 times)

0 Members and 1 Guest are viewing this topic.

Offline WillLem

  • Moderator
  • Posts: 3393
  • Unity isn't sameness, it's togetherness
    • View Profile
[FEAT] Rewind button / hotkey
« on: March 19, 2023, 09:38:16 PM »
I'm currently in the process of adding a Rewind button to the panel. Progress has been slow but significant, and it will likely need extensive bug-testing as it's not only a brand new feature but one which calls upon and interacts with existing features (that weren't purpose-built) for its functionality. So, I feel it warrants its own topic.

Hotkey support will also be provided, of course.

Unsure yet whether to allow this in Classic Mode. Currently leaning towards yes, since the aim is for it to feel much more like a real-time control than a "player-assist" kind of feature, and I think that more people will use Classic Mode if they know that there is a way to undo mistakes, albeit a relatively elementary one. Maybe this decision can be made after we've had a chance to test it out a bit.

Here's what we have so far:

:tal-gold: Fully-functional TTimer-based Rewind mode which performs a repeated -3 Framestep every 50ms.

:tal-gold: Rewind uses backwards Frameskips to function, so Replay-After-Skip mode is cancelled when Classic Mode is active.

:tal-gold: Dedicated panel button & hotkey controls.

Still need to do:

:tal-silver: After implementing this via the TTimer method, button responsiveness is up to about 95% (from about 75%). Still room for improvement, but it's way better.

:tal-gold: Panel display occasionally glitches out (the lem counts and timer flicker between values whilst the Rewind mode is active). This is now fixed as of 2.1.
« Last Edit: May 30, 2023, 06:01:41 PM by WillLem »

Offline WillLem

  • Moderator
  • Posts: 3393
  • Unity isn't sameness, it's togetherness
    • View Profile
Re: [FEAT] Rewind button / hotkey
« Reply #1 on: May 30, 2023, 05:57:52 PM »
OK, Rewind mode has been given a vast improvement for the next release (2.1). It's no longer a GameSpeed, and instead has a dedicated TTimer which performs the -3 backskips every 50ms. Panel flickering is eliminated, and button responsiveness is up to about 95% (from about 75%).

There is still room for improvement: I'm going to try and have the button only set a True/False flag, and nothing else. Then, a dedicated procedure can deal with all of the Rewind mode's various functions based on the status of the True/False flag. This procedure will then get called in Application_Idle so that the game is always checking for the True/False status of the flag. In theory, this should bump responsiveness up to 100%, but this isn't a guarantee - my best guess is that the game is trying to do 2 conflicting things at once, and that's what causes the button to occasionally be unresponsive. As said, though, the TTimer method and associated values mitigate this almost perfectly, so the button is now much more usable even if it can't be improved any further.

For 2.1, I'm also going to trial setting the game to normal forwards speed instead of pausing when Rewind reaches frame 0 (the beginning of the level); I think I prefer it especially since normal backstepping/Restart can already reset to rame 0 and pause the game. It seems a better idea to keep Rewind more of a "real-time play" feature which keeps the gameplay in flow rather than being another stop/start control.

Offline WillLem

  • Moderator
  • Posts: 3393
  • Unity isn't sameness, it's togetherness
    • View Profile
Re: [FEAT] Rewind button / hotkey
« Reply #2 on: July 13, 2023, 05:54:56 PM »
As part of a general need to slow down and stabilise currently-implemented features before taking on new ones, I've decided to stop any and all progress on SuperLemmix until this issue is fixed.

Basically, the Rewind feature just isn't working how I'd originally hoped, and is in serious consideration for being revoked if I can't fix it.

The desired function of it is to be identical to the effect of setting a hotkey to "- 3 Frameskip" and then pressing and holding that hotkey whilst in-game - this is what I want the Rewind button to do! The game is clearly capable of receiving this input, it's just a question of HOW? The current Rewind mode implementation aims to perform this action, but slightly fails.

The problems with it are these:

1) Occasionally, when Rewinding over long periods, instead of performing a (- 3) skip, as intended, it will "jump" backwards several hundred frames. There seems to be a pattern, it can be replicated thus:
  • Load any level with an infinite time limit and no way for the lemmings to die if no action is taken
  • Skip/FF ahead into the level until at least the 3-minute mark (5-minute mark is better)
  • Press Rewind, and keep an eye on the time display
  • At first, it will count down one second at a time: 2:59, 2:58, 2:57, etc
  • After a while, it will start to "jump" 7 seconds at a time: 2:43, 2:42, 2:41, 2:40, (jump), 2:33, 2:32, 2:31, 2:30, (jump), 2:23, 2:22, 2:21, etc
  • The jumps will then become more pronounced, jumping 27 seconds, and then 57 seconds, until it just skips back to the start of the level
Note: Namida has suggested that this could be because the procedure isn't jumping to the correct frame each time.

2) Sometimes, whilst the Rewind button is in the "pressed" state (with the selected graphic drawn, and the Rewind mode in effect), it won't then respond to subsequent button presses, and so won't cancel out of Rewind mode. This is intermittent - it happens about 20% of the time. It also happens when using the Rewind hotkey, so we can eliminate it being anything to do with the mouse, or the mousedown event handling.



Here is all code relating to the Rewind feature. If anyone has any programming experience, I could really do with some help with this. If I can't find a solution, the feature will have to be removed :'(









I'm pretty sure that's all the code relevant to this feature. I have tried testing to see whether the button is setting the flag correctly, and it is. Other buttons do set the RewindPressed flag to false (for example, pressing Pause or FF sets the flag to false in order to cancel the Rewind mode), but even if these lines are commented out, the above problems both still occur.

I have also tried using Message boxes and Debug strings to try and figure out what's happening with the Rewind flags, and to track the CurrentIteration, but so far nothing has helped - this could be because I'm not checking the right things in the right places, though.

I've had these 2 ideas, which between them might help to solve the issue, but I'd rather see if anyone with programming experience has any better ideas before trying these:

2 ideas to fix the issues (click to show/hide)

Any and all ideas, suggestions and help are welcome. I really don't want to have to cull this feature, but if we can't get it working between us, then it will have to go.
« Last Edit: July 14, 2023, 08:45:54 PM by WillLem »

Offline Simon

  • Administrator
  • Posts: 3876
    • View Profile
    • Lix
Re: [FEAT] Rewind button / hotkey
« Reply #3 on: July 13, 2023, 09:05:19 PM »
Quote
At first, it will count down one second at a time: 2:59, 2:58, 2:57, etc
After a while, it will start to "jump" 7 seconds at a time: 2:43, 2:42, 2:41, 2:40, (jump), 2:33, 2:32, 2:31, 2:30, (jump), 2:23, 2:22, 2:21, etc

Hunch: From these numbers alone, the bug sounds like an effect of hitting a next savestate. NL keeps savestates for backwards computation every 10 seconds, and every minute. Then we'd have a performance problem. (Either your computer isn't fast enough, or the savestating intervals are too inefficient.)

Counterargument: If it were a performance problem, it should manifest in vanilla NL, because we all press and hold a hotkey for -1 framestep and I don't remember seeing what you describe..

Quote
The desired function of it is to be identical to the effect of setting a hotkey to "- 3 Frameskip" and then pressing and holding that hotkey whilst in-game

Sounds like you want rewind to behave like fast-forward, i.e., using the function toggles a mode. While we're in rewind mode, we rewind 3 physics updates every graphics frame (or every what?). Is that what you want?

I'd still be surprised then why regular press-and-hold rewinding (= holding a hotkey to rewind one physics update as often as NL gets to show a graphics frame) doesn't run into those jumps for you, but that your rewind mode does. And I'd have to experience the bug myself first; this post is only my initial guesswork.

-- Simon

Offline WillLem

  • Moderator
  • Posts: 3393
  • Unity isn't sameness, it's togetherness
    • View Profile
Re: [FEAT] Rewind button / hotkey
« Reply #4 on: July 14, 2023, 12:23:14 AM »
Hunch: From these numbers alone, the bug sounds like an effect of hitting a next savestate

Yes, this would make sense. It's likely that the "CurrentIteration" isn't catching up before the next -3 Frameskip is being called, resulting in dropped Save States, like this:

CurrentIteration is 364, we press Rewind
-3 skip is performed
CurrentIteration is 361 - all good
-3 skip is performed
CurrentIteration is 358 - all good
-3 skip is performed
CurrentIteration is 322 because Hyperspeed hasn't caught up with the TTimer
-3 skip is performed
CurrentIteration is 319 - technically all good, but not really
etc...

The thing is, this doesn't happen when pressing and holding a hotkey that's set to "-3 skip".

Best guess:

The "press and hold key" method is being interpreted by NL/SLX as a repeated input, and so it always responds. Sometimes, it lags a little bit (for example on very large levels or when trying to record in OBS to show this feature in action!), but for the most part it is always hitting the beats, and the rendering is catching up in between each one, for better or worse.

With the TTimer method (which is what Rewind mode uses), the TTimer is firing every 60ms regardless of whether the rendering has had a chance to catch up. So, it's always skipping back from whatever iteration (frame) the engine has managed to skip to in that time.

If this is indeed the case, then what's needed is a way of saying to the TTimer "don't update until the rendering has had a chance to catch up!" - I'm not even sure if this is possible...

Sounds like you want rewind to behave like fast-forward, i.e., using the function toggles a mode. While we're in rewind mode, we rewind 3 physics updates every graphics frame (or every what?). Is that what you want?

Yes, that's exactly it.

Essentially, Rewind mode should perform a -3 frameskip, then another, then another, as close to 60ms apart as the graphics rendering will allow. If we get a little bit of lag here and there, that's infinitely preferable to the savestates not updating quick enough.

I've made a video to help show what's desired from Rewind mode, and what's actually happening. The unresponsive button presses are enough alone to warrant the feature being culled, so that's a must-fix.

Offline WillLem

  • Moderator
  • Posts: 3393
  • Unity isn't sameness, it's togetherness
    • View Profile
Re: [FEAT] Rewind button / hotkey
« Reply #5 on: July 14, 2023, 01:17:52 AM »
Great news! I've figured out what was causing the button to occasionally be unresponsive!!!

The answer lies in the actual mousedown event procedure itself. Here, we can see that the entire mousedown procedure is exited from whenever the game is in hyperspeed!:

Code: [Select]
procedure TBaseSkillPanel.ImgMouseDown(...);
...
begin
  if fGameWindow.IsHyperSpeed then Exit;

  // Rest of code for skill panel button presses...

What this means is that, whenever the game is in the state where it's hyper-jumping forward to (CurrentIteration -3) as part of the Rewind procedure, the panel won't respond to mouse clicks! Whilst this is necessary most of the time, it's not ideal when attempting to exit out of Rewind mode. So, this needed to become:

Code: [Select]
procedure TBaseSkillPanel.ImgMouseDown(...);
...
begin
  if fGameWindow.IsHyperSpeed and not Game.RewindPressed then Exit;

  // Rest of code for skill panel button presses...

This fixes the button unresponsiveness issue. The same has also been done for the Rewind hotkey, which relies on the keydown event handler.

We're halfway there!!!!! :lemcat:
« Last Edit: July 14, 2023, 01:23:00 AM by WillLem »

Offline Simon

  • Administrator
  • Posts: 3876
    • View Profile
    • Lix
Re: [FEAT] Rewind button / hotkey
« Reply #6 on: July 14, 2023, 04:53:11 AM »
Thanks for the video. Right, it's not a classical performance problem, the difference is too glaring indeed between NL's regular -3 rewinding and rewind mode. It's possible that performance still enters the equation in how soon/how far you overrewind, but the root of the problem is likely elsewhere.

Good find with the strange UI/rewinding interaction. Reminds me of how I sometimes can't right-click scroll during NL rewinding.

The jumps relate to the NL savestating pattern (every 10 seconds) in the video as strongly as you've described. Next idea: I'd drill into the savestate-retrieving logic. E.g., what program state persists in the game class after stateloading, but makes no sense anymore after stateloading?



Unrelated to the immediate rewinding bugs:

Disabling UI during hyperspeed looks like a fertile ground for bugs. Why does NL call into UI handling during hyperspeed in the first place? Trick to make NL more responsive? Or a design relic from when UI and physics were too intertwined from the L1/Lemmix days? The whole hyperspeed idea permeates the game code and it feels strange in many places. Physics, UI, ... shouldn't rely on how fast we are computing physics updates.

If you have time, rip out the entire early exit (if Hyperspeed Exit), see what bugs come up, and design them out of existence another way. Feels like a deep rabbit hole that will redesign half of the program. Even if you don't merge such explorative changes, you might learn more about ye olde golden clockwork.

-- Simon

Offline WillLem

  • Moderator
  • Posts: 3393
  • Unity isn't sameness, it's togetherness
    • View Profile
Re: [FEAT] Rewind button / hotkey
« Reply #7 on: July 14, 2023, 06:26:16 PM »
Good find with the strange UI/rewinding interaction. Reminds me of how I sometimes can't right-click scroll during NL rewinding.

Yes, that would probably also be a result of the Hyperspeed exit. If you think it's worth removing it to see how the program behaves without it, I'm all for that.

drill into the savestate-retrieving logic. E.g., what program state persists in the game class after stateloading, but makes no sense anymore after stateloading?

Can you be more specific about what you mean here? Maybe we're due another screen-sharing session - I'd be happy to show you exactly what I'm looking it, it'd be good to work with a bit of guidance.

Anyway, we're in a good place - the UI unresponsiveness is now eliminated for Rewind (and, by extension, the new Turbo mode which was also suffering from the same unresponsiveness as a result of its reliance on Hyperspeed), so the only problem that remains is the unsynchronised skipping.

I put the question to OpenAI, and the suggestion I got back was to implement another timer which pauses the Rewind timer long enough for it to wait for the rendering to update. Something along the lines of:

1) Rewind is pressed, Rewind Timer is activated
2) Current iteration is stored as "previous iteration"
3) -3 Frameskip is performed
4) Rewind Timer is deactivated
5) Rendering Timer kicks in and checks for the condition {the current iteration is exactly 3 less than the "previous iteration"}
6) As soon as that condition is true, the Rewind timer is activated again
7) Repeat from step 2

Essentially, the Rewind Timer would be stop-start activated by the Rendering Timer, which makes it wait until the game rendering has caught up rather than indiscriminately calling GoToSaveState every 60ms (which is, I'm about 88% sure, what's causing the SaveState drops).

This seems like quite a drastic and potentially CPU-intensive way to handle it though, is it advisable?

Offline Simon

  • Administrator
  • Posts: 3876
    • View Profile
    • Lix
Re: [FEAT] Rewind button / hotkey
« Reply #8 on: July 14, 2023, 06:59:54 PM »
Sorry, I don't know enough details to give anything better than these high-level research ideas.

I'd be up for an hour of code-plowing on Mumble. Maybe Sunday, July 16th, at 09:00 UTC?

-- Simon

Offline WillLem

  • Moderator
  • Posts: 3393
  • Unity isn't sameness, it's togetherness
    • View Profile
Re: [FEAT] Rewind button / hotkey
« Reply #9 on: July 14, 2023, 08:33:07 PM »
I'd be up for an hour of code-plowing on Mumble. Maybe Sunday, July 16th, at 09:00 UTC?

Sounds good! :thumbsup:

I'll give you a shout on QuakeNet either way.



Meanwhile, I think I've found a solution!

It comes down to hyperspeed again. After analysing debug output for the value of "CurrentIteration" at various different parts of the Rewind process, I was able to ascertain that the game was indeed firing off additional backskips at times when hyperspeed hadn't had a chance to catch up.

So, I've implemented this check in the main Application_Idle method - whilst Rewind is pressed, this check makes sure that the Rewind Timer isn't enabled whilst the game is in hyperspeed. It's essentially a "wait for everything to catch up before jumping back again" check - much simpler and more elegant than any proposed double-Timer acrobatics or Rewind mapping (neither of which worked anyway, incidentally!):

Code: [Select]
  // Rewind mode
  if Game.RewindPressed then
  begin
    SkillPanel.DrawButtonSelector(spbRewind, True);

    if GameParams.ClassicMode then
      Game.CancelReplayAfterSkip := true;

    // ensures that rendering has caught up before the next backwards skip is performed
    if IsHyperSpeed then
      RewindTimer.Enabled := False
    else
      RewindTimer.Enabled := True;

It works, and it's a good enough solution that I'm happy to include it with 2.4.2, which I'll release later this evening.

However, there is some lag and slowdown with this method - it's not feature-breaking, but it's noticeable. It's the least of my worries though, given what the Rewind feature has had to content with so far - flickering panel display, unresponsive button & hotkey, dropped save states, etc... given it's short history, it's now the healthiest it's been so far!

Fix 1 in Commit 5c1c02e97
Fix 2 in Commit d91f57053


The question remains, though - why are we able to perform repeated backskips by pressing and holding a key, without any of the aforementioned issues?

bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb

It responds smooth as butter. Why can't Rewind mode also do this?