You could use Lemmix.
https://www.neolemmix.com/?page=download_list&program=42Lemmix is a clone of DOS Lemmings, made for Windows. Its most notable feature is that it reproduces the physics of the DOS games
perfectly (or at least, there are no known differences - and trust me, this has been examined to an
extremely deep level). It does have some extra features like rewind, replay and fast forward, but you can choose to simply not use those if you want an authentic experience. And yes, it does keep the correct music for the level.
The one difference that's blaringly noticable - it has the Amiga musics rather than the DOS ones. You can fix this by downloading the "Lemmix DOS Musics" on the same page and placing them in the same folder as the Lemmix EXE. They're not included by default for file size reasons - Lemmix can natively use Amiga's music format, using the original 10KB or so files. It can't natively use DOS's format, it had to be converted to OGGs, which are relatively large (they're generally slightly smaller than an equivalent MP3).
That aside, DOS level codes don't work on Lemmix and vice versa. Instead of remembering level codes, just enter "CHEATCODES" as a password. Then you can enter eg "TAME15" as a password to go to Tame 15. This way, you only have to remember which level you're on.
(If you aren't overly concerned about having an exactly DOS-like gameplay experience, consider NeoLemmix instead. It's a fork of Lemmix which does away with "replicate DOS perfectly" and instead fixes all the glitches, and adds a bunch of new features (as NeoLemmix's primary intention is for custom levels, but you can play the official games on it too), while retaining a DOS-like visual appearance for the most part.)
If you insist on authentic DOS ONML, as far as I know there's no music-fix version (though someone else might correct me here). One workaround you could do is, before you start recording, go solve the first few Tame levels (off-camera) until you're up to the correct music.