That is a nice example.
Cyan: Yes, the bones are so prevalent there that it's usually a mishap when we click in their small hollow gap, expecting the bones to be selected, but the dirt would be preferred by pixel. Selecting the dirt here is annoying no matter what.
The best justification here is that Lix already highlights the hovered tile (that isn't yet selected) with a darker rectangle (hover is grey, selection is white), and you would see that the wrong tile is hovered before you click. You would instinctively move the mouse slightly to hover the bones, then click.
I am used to this visible hovering, can't tell how nasty it is without. I'd expect it to be much nastier without visible hovering. But I also can't tell either how often this mishap happens of clicking in the tiny hollow space. I'm personally weak with mouse pointing in general, I often misclick stuff even in non-Lemmings GUIs.
Green: My prevalence is weak, too, but merely because the dirt is so large; it feels as if the user should point more clearly to the dirt to select the dirt. Still, it sounds reasonable to program the green spot to select the dirt. I wouldn't be annoyed if it selected the bones.
Yellow: This is much more clearly a spot on the single bones. I might get annoyed if it picks the dirt, but at least the annoyance would be over quickly because I'd immediately re-click with priority invert. Priority invert in general is super useful in any editor here, and it's good that both editors have it.
Thanks, I see that both algorithms have their drawbacks and nothing does 100 % what the user thinks. It might even depend on what tiles are common in the games.
If we were to optimize this to the extreme, maybe tile designers should tell the game how they want each tile to be treated: Selectable by convex hull (bones) or by pixels (mesh).

-- Simon