I prefer double quotation marks for speech in fanfiction and singles for speech in handwriting or published novels, but I think that's just because it's what I'm used to in each case. If it's not speech, though, doubles look incredibly odd to me no matter what the format.