Personally, I'd see Sam as more of a bottom, but Dean as the one who'd have the biggest issues with incest. Like you said, Dean seems like the one who'd have the most issues with masculinity and heterosexuality, and have the hardest time dealing with being penetrated.

I'd see Dean as having a harder time with the incest thing precisely because of the potential to hurt Sam; Dean's the older brother, and I have the impression that he did a lot of looking after Sam when they were growing up (John doesn't strike me as a guy who'd provide regular babysitting and make sure the kids had lunches ready every morning). Which means him initiating or participating has much more of an air of abuse, or at least worrying power imbalances. Whereas Sam, the kid brother, isn't going to feel like he's pressuring or taking advantage of Dean as much.

Sort of how I see family dynamics. I get what you mean about Dean's Winchesters Are Different thing, and Sam not buying into it as much. In my mind, the potential for abuse of power, and Dean worrying about what he might be doing to his brother, outweigh that.