It kinda goes back to society's expectations of manliness - basically, men traditionally define themselves by what they are NOT, which are usually "feminine" characteristics. Also consider the assumption that women are passive recipients of sex (fuck-holes), and a man is less a man if he himself is a 'fuckhole'. (Homophobia is really rooted in woman-hating; gays = less than men = women = ew.)

And of course, manly men who are homophobic bigots don't like the idea of being fuckholes, so the idea of a man being attracted to THEM and rendering them fuckholes is a bad bad BAAAAD thing.