I have asked many regular non-philosopher type folks about how to avoid
It's an interesting question and not easy to answer. Let's start with what may seem to be a minor point but actually isn't. It's not right that we should never be direct. The most obvious exception is when two people already have a sexual relationship and they're both comfortable about it. But even there, being blunt isn't always welcome. Sex isn't one-dimensional. There's lusty animal sex and there's also tender romantic sex. What works for one doesn't necessarily work for the other.
If it's complicated even for people who are in a relationship, it's not hard to see why rude and crude doesn't tend to work when that's not so. Human relationships just are complicated; after all, there are completely non-sexual matters that most of us don't like having broached too directly. When we add sex to the mix, things certainly won't get simpler.
Leave male vs. female aside for a moment. If someone hints to me that they're interested but the feeling isn't mutual, I can ignore the hint in ways that get the message across but don't hurting the other person's feelings or make them lose face. This doesn't go just for sex, but it seems safe to say that it goes particularly for sex. Being less direct can make things a lot less awkward.
A different sort of case might help. If I'm upset with someone, then depending on the relationship and the reasons, being clear and straightforward might be best. But the old saying "least said, easiest mended" often has a point. A certain amount of indirectness seems to make social life easier.
The fact of the matter is that there's a lot of communication that doesn't take place using words, and on the whole, we humans seem to like it this way. The advantage is that this adds a lot of nuance and subtlety to the way we communicate. But not everyone is equally fluent in the language of gesture, gaze, tone of voice and standard dictionaries are hard to find.
As noted, all of this is general and applied to a lot more than sex. But there's another issue here that's at least as important. There's often a lot at stake in sexual encounters, and there's usually a lot more at stake for a woman than for a man. For the most part, unwanted sexual attention isn't a problem for men. For women it very often is. At the very least, staying away from the rude and crude is a way of acknowledging that important fact.
Afternote: a friend pointed out this very instructive youtube video in which Steven Pinker says a lot about all these issues. Stick with it to the end.