This has happened to me, even though at the time I was young and conventionally attractive. I have had friends over the years that look like they are more likely to have sex with a stranger than I am. When that is the case, men ignore me and/or are rude to me as they're trying to get in my friend's pants. It's honestly so gross. And it's gross in both directions. If you can't be a decent human being to someone you don't find attractive then you definitely need to work on self development.