It's definitely a negative, and I admittedly didn't start riding a bike again until after I had moved here 10 years ago (maybe bike access to downtown would've factored into my decision if I had already been biking).
I do think that sometimes too much can be made of the lack of a bike/ped connection though. If you live near Maverick you can be in downtown in one subway stop. And for all but two hours a day you can bring your bike on the T. My guess is that people living in Watertown or Malden (both places I like a lot) are much less likely to find themselves on a bike or on foot in downtown within 15 minutes of leaving their house than I am. Certainly downtown isn't the center of gravity for every activity (Cambridge, Fenway, etc), but other than the late night cab issue (soon to be mitigated by later subway service) Eastie is more convenient to places like the North End, Faneuil Hall, DTX, Financial District, the Common than any other neighborhood that's not in downtown.
This is where I think the barrier is more psychological than physical (as another poster mentioned). Most people in watertown and non-core neighborhoods drive or take the T to work. Biking and walking are still a rarity (especially for the watertown/malden group- I'm not saying its not a plus, but only committed cyclists do that to the point where its a deal breaker).
I think people do focus on the exceptions rather than the rule and that this will change in Eastie. It's the same reason people buy houses with giant extra spaces "for visitors" which may be 2 times a year, when it would be cheaper to just buy people a hotel room.
Eastie is very isolated in the Boston mentality. It's a walking city. When I lived in the North end, I would get up and meander and end up in Fenway. I never went to Eastie. Now I am in Somerville (connectivity to a job in cambridge was primary for me- looking at you red-blue connector!), but if I were downtown it would be close. Rents along the red and orange corridors (to say nothing of the corroding trains) are beginning to push people to consider eastie. In ten years, we will have posts talking about how yuppie coffee shops and fake pizza places are pushing out established institutions and immigrant and workforce housing.