Before you can load or save replays you must first have loaded or saved your game. On the main menu screen there is a candle next to the lemming playing the bongos. Use this to save or load your game. This game does not auto-save, so make sure to save your progress when you stop playing.
Once your play session is associated with a save file (by loading or saving) you can now also use the replay feature. To save replay of the level you just completed, click on the save replay button after beating a level. To load a replay, you right-click on the dot that represents that level and the replay will be loaded if you have one. Replays are specific to the save file, so each save file will have its own replays.
Note that ridiculously, if you solve a level using the replay feature, either by loading a replay or pressing R during gameplay, it does not count. You can only unlock the next level by beating it without using replays. Why they decided to do this I will never understand.
If all you want to do is keep ypur progress between play sessions, loading and saving on the main menu is all you need and replays can be ignored.