Here's a plausible principle: in general, we shouldn't do things that are likely to make people uncomfortable. This is particularly true if our only reason for doing whatever we're doing is that we get some sort of enjoyment out of it. And if we're in doubt about whether we're likely to make the person uncomfortable, better to err on the side of caution.
The principle is actually a broad one, as we can see if we change the example a bit. Suppose the person sitting across the room from me has a very sweet face. There's nothing wrong with noticing, but staring is another matter; that's likely to make the person uncomfortable. This is true even if the s/he has made some effort to highlight facial features. Noticing, even appreciating is one thing; staring, let alone ogling, is another.
That's the general advice. In real life, there are lots of subtleties. It's not unusual for one person to notice that another is "checking them out," as it's sometimes put, and to be flattered. That might be particularly true if the setting is a bar where people go to meet one another. But even there, the general rule is still a good one.
Maybe the simple version is this: I shouldn't be creepy. And if someone might well think what I'm doing is creepy -- even if I don't mean it to be -- I shouldn't do that either.