Peter Capaldi is utterly brilliant, but the persona, the relatively straightforward "grumpy old man" will probably never make it to my favorites list. His introduction was also the one I found the most troubling to date, in the sense that it took him so long to move past the forgetfulness phase just after regeneration that it reminded me unpleasantly of relatives with dementia. I think that might have actually been what they were going for, it just wasn't for me. 

The fourth Doctor is the most iconic, most charming, most whimsical. Tom Baker appearing at the end of the Day of the Doctor might have been my favorite moment of the revival series. 

I never cared for Tom Baker even after watching some of his work.

No, it wasn't that since I like Matt Smith being a goof ball. It might be because everyone holds him so high disregarding all the new ones that I don't want to recognize him at all.