The reason this happens seems to be because the lemming arrives as a faller, therefore moves downwards (and thus, out of the portal's trigger area) before transitioning to a swimmer. That this happens feels like a bug in and of itself, and fixing that should also correct this bug.
Additionally, all evidence suggests this bug would also happen with teleporters, so will be fixed for those in 12.13.X regardless of what happens with the portal.
Fixed for teleporters in commit 6856bfc, and then for portals in commit 27e5c3c.