Replay VerificationYou can run Lix noninteractively on a directory full of replays:
- lix --verify mydir will tell you, for each replay, whether it passes its pointed-to level. Finally, Lix tallies passes and fails.
- lix --coverage mydir does everything from --verify mydir, and will also list the uncovered levels, i.e., levels in a level directory into which at least one replay pointed, but that themselves had no passing replay.
It's really a single feature with two forms of invocation. You want
--coverage when you expect most levels to be covered by your replay collection and want to find omissions. Otherwise, you want
--verify, which also makes sense for individual replay files.
The origin of replay verification: Around 2010-2013, geoo wanted to maintain online highscores for Lix. He considered to write a server based on Django, let people upload replays, then have the server run Lix in the background to verify the replay.
Thus, from day one, I saw a clear need to make replay verification run with as little of Lix as possible. I would still have to initialize graphics to load tiles, but I wouldn't be able to run the graphical user interface on the server. Everything had to work from the command line and print results as text output.
In the end, we never had online highscores. Nowadays, I don't even want them, they're extra stuff to maintain. There are enough other juicy objectives for Lix development in the next few years. But even without online highscores, the replay verification enjoys fast runtimes. It doesn't have to maintain a graphical land, only a physical land.
The biggest use case of replay verification is not competition between level players; it's tooling for level authors to verify pack solvability. I don't remember when exactly we realized this use case. Maybe we always had it in mind and it merely became the main use case over time.
Fully covered packs serve me as physics test cases: Whenever I touch physics code, if all replays pass, then most likely, I haven't changed physics.
I believe we had cross-pollination between Lix and NeoLemmix about replay verification, but I forgot the details. Most directly comes to mind: I'm still worrying how to associate replays to levels. Neither (level path only) nor (NL's level ID only) is perfect. And we eventually must start classifying replays in the proof collections: Intended solutions, accepted alternatives, backroutes, ...
-- Simon