It's because developers are focusing too much on high-end residential market or high profit niche residential market. The average family can't find affordable housing because developers are building micro-units that doesn't have room to support a family of four or are building luxury condos that a family of four can't afford. An average family of four aren't looking for a condo in a tower that has a gym, a swimming pool, and a lounge which ups the cost per unit. They are just looking for a regular apartment.
What is wrong with building these units if they are bought and used?
Also, I live in a condo building with these amenities and I see lots of parents with strollers, young children, and teenagers. There is some foreign ownership of units, but they are all rented out; vacancies in the building usually only correspond to renovations and investors trying to sell units letting them go vacant to widen the pool of potential buyers.
When these units aren't built, the wealthy who want to live in Boston often still buy, just in the existing housing stock. Having them buy in the luxury buildings does lower the pressure on prices in those buildings. Wealthy people have no difficulty buying up units and renovating them to live in. They'll even buy two adjoining $500K units and combine them. That sure does wonders for housing affordability, reducing the number of available units and doubling the price!!