When adjusting mirrors, I want to get as much blind-spot visibility as I can. I therefore have my mirrors turned pretty far out, so far that I can just see the tips of my elbows on the inside edges. I do have to move my head to see behind me, but I personally find the sides to be more vulnerable areas.
I know what you mean though. Bike mirrors tend to be pretty crappy relative to those in cars in terms of total visibility.