Madrid was one of the best I've visited (never been to Tokyo). There was absolutely nowhere I couldn't get to via subway. It covers the metro area well far beyond the core of the city. It probably loses points because it's not nearly as large as Tokyo, Paris, London or New York. Still, I don't know if I've been to a city where public transportation was as extensive and as easy to use. I never once thought, "well maybe I'll get a cab" or "I wish I had a car for this." Those thoughts have come to mind more than once in New York and London and just about every other city I've been to.
Madrid gets my vote.