How did Boston support a population of 800k people in the 1950s but would struggle to support 20% more than the 685k we have now?
Honestly: blame birth control.
The 1950s population had WAY more children (and stay-at-home moms) than today's population does. Even though our total population is lower than it once was, I believe our adult working population is at an all-time-high. Back then the typical Boston triple decker might hold 15 (or more) people, of which only 3 or so worked outside of the home. Now that same building might hold 9 people, of which 6 or so work outside of the home. (Obviously these are made up numbers, but you get the point). Kids and stay-at-home parents don't take up very much transportation infrastructure; commuter do. Compare Southie today to Southie of old days; the total population is much lower today, but all the old public schools are empty and the streets are clogged with condo-living professionals (most of which have no or few kids) trying to get to work. Today's norm of double-income-few-child households results in lower total populations per unit but much higher requirements for transportation.
Also, while Boston's total residential population has fallen, Greater Boston's overall population (and especially commuting population) has continued to rise. So there may be fewer people living in Boston today then there were in the 1950s, there are more people physically in Boston today then there were then. By some measures, Boston's daytime population on a typical workday currently exceeds 1 million.
So in short, fewer children + more workers = lower total Boston population but more people on the streets.
-----------------------------------
This is a reason why I see the frequently cited solution of increasing housing in ex-urban areas with improved commuter rail as lacking. Arguably, people
living in one's neighborhood negatively affect quality of life
less than people
commuting into one's neighborhood do. I live in the Fenway because my partner works crazy hours in Longwood. By living close to the hospital she can just walk to work every day. Rush hour traffic in my neighborhood is a mess not because of the people like her who live in the neighborhood and walk to work, it's a mess because of all the commuters who don't live nearby. Building more housing nearby and allowing more people to walk to work wouldn't negatively affect my quality of life nearly as much as pushing more people to commute in from afar would.