As a former downtown resident, I don't find driving downtown to be particularly stressful. You need a mental map of where you're going and need to just accept that for many parts of the day, you're probably going to be sitting in traffic. You gotta just go with it.
That said, I much prefer...