"Take Up Archery", from Lemmings 2, is the hardest and most annoying level of all the saga. I'm sure most players would agree with me 

Yup, most definitely! Ugh, I remember the numerous restarts because I couldn't aim the arrows correctly.
Another L2 level I want to add is Highland 4 (
I think it's Highland 4?), "A mere stone's throw." where you have to make a ramp with stones/rocks. It's a nice concept and all, but it's so annoying to get right without getting anyone trapped.
nobody has mentioned "How Do We Get Up There" from Lemmings Revolution:
The worst kind of annoying; the devs took a concept that seems good on paper, or at least has potential then ruined it.
Sadly the ridiculous time limit completely ruins what would otherwise be a good level. What makes this level so hard is not only the time limit but with time limit removed altogether, I argue it would still be decent challenge. It looks pretty straight forward but turns out more complex than it at first seems, getting multiple worker lemmings where you need to go, a lack of skills and saving 100% isn't trivial by any means. I remember, back in the day being stumped on this level for a while before figuring out the proper route... BEFORE even realizing the problem of the time limit.
Completely agree there mobius! It's a great level and all, but the super tight time limit makes it so annoying that I dread having to do the level again after finally getting everything right, only to have to do it again because I ran out of time.
Another Revolution level I can add: I think it's called "Don't let them out!" If you remember, it took me nearly 2 hours on camera before I finally got it solved! At least when you're trying for what I believe is the intended solution to the level. Again, while you have more time available here than the level you mentioned, we still have a super tight time limit and the execution is so nightmarish to pull off on the level. Not only that, there's also a lot of good timing needed and even the annoyance of the bug of a 50:50 chance of either surviving or splatting, and on top of all of that one of the spinning gear traps has a really huge hitbox that makes it so hard to avoid by building over it. All with a very limited skillset that nearly provides no room for error.
Video of me solving the level with what might be the intended solution:
https://youtu.be/uNevVNqwnL8We All Fall Down.
The solution is of course trivial, having been designed for the original game as an extreme example of a level where doing the solution is where the difficulty comes from rather than figuring it out. But from that regard, really only the Mayhem and maybe the Taxing versions are "interesting" in the sense that they may require a slightly different solution just to be reasonably completable, although NL framestepping tools show that the level DOES actually have enough space to do the trivial solution, if you can somehow pull it off without those player assists. And ultimately the thing that you have to do to be able to pass it isn't very interesting; it's just finicky because it's easy to have to start over because of a single very slightly mistimed input.
The interesting thing is that these levels are completely trivialized in ports where you have the increased fall distance before splatting, as you can win by doing nothing in this case, since they'll survive the fall without needing to dig. You can also avoid the challenge altogether in ports where digger cancelling is possible. In NL, for example, it's possible to complete the level using only two diggers!
If anything, I feel like the challenge could had been increased further at least for the Taxing and Mayhem versions by providing you with way less diggers than there are lemmings for those levels. Like probably half the number of diggers or even less. Maybe considerably shorten the diggable platform even.