People often find it odd that I generally drive sockfooted. I'll admit that it is uncommon. I find driving with shoes on to be terribly awkward, though, now that I've gotten used to driving without them. It feels so clunky and it just doesn't seem like I get nearly the level of control I want when I'm wearing shoes. I'd liken it to driving while wearing two of those giant foam hands: you could do it, but it's clearly superior to have more physical contact with the steering wheel.
I recommend trying it out sometime. It's especially easy for me since I wear sandals when possible, and it's much easier to take those off than shoes. It felt sort of strange the first time I tried it (I was 16 or so), but since then it's always felt natural and "right." (Some people I've talked to thought that it was illegal for some bizarre reason, but it is perfectly legal everywhere in the US and Canada.)