You're probably right, and it's unfortunate. I think if men had the belief that it is better to be alone than with the wrong person, they'd find a more fulfilling experience.
Even for purposes of sex, which I know men find as their biggest motivating factor, there are many women who will never be sexually compatible with you.
It truly is the case that being happy and fulfilled alone will allow another happy and fulfilled person to find their way into your life, which creates a much healthier, more stable relationship.
If you are constantly with the wrong person, you will never find the fulfillment you seek.