16
NeoLemmix Main / Re: Don't exit on losing all lemmings (feature development)
« on: April 18, 2024, 04:13:39 AM »
Thanks. I'll call that executable nl-2024-04-08.
Icho has sent me his replay coverage for Lemmings United. I've looked at his 208 replays for what is not in the bonus rank of United. These 208 replays are mostly 1 per level, sometimes 2 per level.
I've ran those 208 Lemmings United replays separately through each of the following:
All 208 replays pass (solve their level) in all of the 4 runs.
Your nl-2024-04-08 produces output independent of the 3-way option. In more detail: When I run mass replay verification in the nl-2024-04-08 with the option to Exit if Save Requirement Met, I get the same output (i.e., identical text file) as when I run mass replay verification in the nl-2024-04-08 with the option Never Exit to Postview, or with the option Always Exit to Postview. This is good.
Your nl-2024-04-08 produces different output than NL 12.12.5. In NL 12.12.5, most (all?) replays run for exactly 1 physics update longer than in nl-2024-04-08. It's drudgework to check this claim for all 208 lines, I haven't written a script to verify that claim for me.
For example, NL 12.12.5 produces:
nl-2024-04-08 produces:
The only difference here is that NL 12.12.5 runs for 1882 physics updates ("frames") and nl-2024-04-08 runs for 1881 physics updates before both conclude that the replay passes its level.
We can explain this with the reworked end-of-level behavior: NL 12.12.5 must start a new physics update before it can conclude that the map is over. nl-2024-04-08 can test for completion in between two physics updates. Therefore, nl-2024-04-08 needs one physics update fewer. Thus, I believe: There is nothing to worry here. Do you agree?
The high-level result (pass or fail) is identical across NL 12.12.5 and nl-2024-04-08. All levels pass.
This doesn't conclude the testing of mass replay verification yet. Reason: I've only tested solving replays. I haven't tested a big bucket of failing replays, of indeterminate replays (that run for too long after final skill), of weird nukes in the replays, ...
And I have replays for 4 more Icho packs. That's for next week.
-- Simon
Icho has sent me his replay coverage for Lemmings United. I've looked at his 208 replays for what is not in the bonus rank of United. These 208 replays are mostly 1 per level, sometimes 2 per level.
I've ran those 208 Lemmings United replays separately through each of the following:
- NL 12.12.5 stable,
- nl-2024-04-08 with the option: Always Exit to Postview,
- nl-2024-04-08 with the option: Exit if Save Requirement Met,
- nl-2024-04-08 with the option: Never Exit to Postview.
All 208 replays pass (solve their level) in all of the 4 runs.
Your nl-2024-04-08 produces output independent of the 3-way option. In more detail: When I run mass replay verification in the nl-2024-04-08 with the option to Exit if Save Requirement Met, I get the same output (i.e., identical text file) as when I run mass replay verification in the nl-2024-04-08 with the option Never Exit to Postview, or with the option Always Exit to Postview. This is good.
Your nl-2024-04-08 produces different output than NL 12.12.5. In NL 12.12.5, most (all?) replays run for exactly 1 physics update longer than in nl-2024-04-08. It's drudgework to check this claim for all 208 lines, I haven't written a script to verify that claim for me.
For example, NL 12.12.5 produces:
Code: [Select]
Pacifism 8: 10000 B.C..nxrp (1882 frames) LvV 0000000000000000 / RpV: 0000000000000000
nl-2024-04-08 produces:
Code: [Select]
Pacifism 8: 10000 B.C..nxrp (1881 frames) LvV 0000000000000000 / RpV: 0000000000000000
The only difference here is that NL 12.12.5 runs for 1882 physics updates ("frames") and nl-2024-04-08 runs for 1881 physics updates before both conclude that the replay passes its level.
We can explain this with the reworked end-of-level behavior: NL 12.12.5 must start a new physics update before it can conclude that the map is over. nl-2024-04-08 can test for completion in between two physics updates. Therefore, nl-2024-04-08 needs one physics update fewer. Thus, I believe: There is nothing to worry here. Do you agree?
The high-level result (pass or fail) is identical across NL 12.12.5 and nl-2024-04-08. All levels pass.
This doesn't conclude the testing of mass replay verification yet. Reason: I've only tested solving replays. I haven't tested a big bucket of failing replays, of indeterminate replays (that run for too long after final skill), of weird nukes in the replays, ...
And I have replays for 4 more Icho packs. That's for next week.
-- Simon