The downtown seems pretty dead anytime I have walked around there (save before a big event at the used-to-be Centrum, now whatever). I guess it's about as uninspiring as most New England downtown areas. Still, I would rate it way behind Providence, Portland, and New Haven, and behind Lowell and even Salem in MA. You could argue that New Bedford has a better downtown area than Worcester.
Now, Worcester is over 38 square miles and I don't know much of it beyond downtown, although once ended up on some major road with a bunch of bars (perhaps part of Route 9?). When you speak to Worcester's vitality, are you referring to downtown or some other part of the city?