In my mind for dorms. The area I keep thinking about is the Harvard-owned area. That's the ideal spot in my mind for an Olympic village. One can build as dense as anyone wants and won't overwhelm as there's nobody to overwhelm. Heck, it doesn't have to do the modular thing, it can be then sold (or built with aid) to Harvard as new dorms with some as new residential towers where it density won't affect Allston. BU would probably be interested too.
The IOC won't approve a proposal where dorm facilities are re-used for athletes, and the reason is twofold: ego and security. I don't think the IOC is desperate enough for the US to host an Olympics to keep athletes in college dorm rooms if Paris, Qatar or Istanbul is willing to build all new and shiny. The quality of the purpose-built Sochi housing might change the perception that new is better (StuVi at BU is certainly nicer than the Sochi village), but I wouldn't count on it.
The IOC also wants a very controlled environment for athletes, in part because of Munich and in part because of the natural concerns of having a lot of high-profile people in one place. A campus with controllable entrances and exits and a non-integrated plan - Wellesley College, say - could theoretically satisfy them, but forget about selling them on Harvard or BU. That doesn't mean the dorms are useless, however, as they'll be incredibly important for housing volunteers, the massive temporary service workforce and members of the press (again, see Sochi for how bad that can get if it get away from you).
I've thought for a while that using Seaport Square or NorthPoint as a temporary village is an idea with some promise. If you can tell the IOC that the rooms will have to be nice since they'll be sold as semi-luxury condos after the games, and that the development has solid financing and a good business case anyway, that might be more compelling than prior efforts to sell them on Athletes' Village cum condo developments (Chicago, anyone?)
Generally, I don't like modulars. For anything. Basically, a modular is a permanent building which is built to be picked up and moved because you need the land. Since it's meant to be permanent somewhere else, it's not inherently less expensive to construct, and it can be more expensive because it has to be designed for disassembly and reassembly. Plus, it's a bunch of labor and negotiations to move all that stuff after the fact - NIMBYs refusing to take it and such.
In general, things should be permanent (the village housing) on a sensible site or fully temporary. John Fish listed 3 venues that Boston doesn't have - a big stadium, a swimming center and a velodrome - to which I'd add a tennis center. Atlanta got Georgia State to build their swimming center for them, and between NU, Harvard and MIT somebody must want one. The other 3 are going to have to be temporary facilities, likely on a single big plot, and the Harvard/Beacon Park land is, IMO, the best place for them. Of course, that doesn't address the other temporary venues every host has to build, like a beach volleyball stadium or a whitewater course (though I wonder if they'd accept doing that in a real river in Western Mass or New Hampshire...).
This brings us to the infrastructure. Don't even start trying to tie in things that don't serve the Games directly, as that will lead to allegations of corruption and significant waste. Even if SCR wasn't a waste in general, it would be a waste as a tack-on for the Olympics (the guy at the meeting seems to have just recycled some MassDOT planning slides that listed all their planned projects). The highway projects (93/95 on both ends and A/B) were mentioned because MassDOT has now fully committed to doing them by 2024. They don't need the Olympics, they'll happen anyway, as will SCR, DMU-on-Fairmount and GLX.
Focus your energy on things that actually need the help. If you put the stadium at Beacon Park, you can tie in DMUs to Riverside and South Station expansion. If you put stuff at Suffolk Downs or Wonderland, that might get you Red/Blue Connector, but probably not Lynn. Roslindale Orange might be sellable if you tie it to events at NU near Ruggles. Hosting lots of preliminary soccer in Foxboro could be an impetus for double-tracking and permanent service to Patriot Place (and perhaps for some Route 1 capacity work).
Bear in mind also that part of having a successful and admirable Olympics in Boston is placing events in facilities we have near infrastructure we also already have. It makes more sense to place BMX or Whitewater near Wachusett than in NH, for example, because the CR line is already in place to get spectators and press out there. If you place a venue in a place where a project MUST happen for it to function, don't think that means the project will actually happen. All you'll get is a big SNAFU when the project fails.
