Haven taken both the silver line and the blue line (with a walk from S. Station to either State or Aquarium preceeding the blue line trip) at various times of the day between once and 4 times per week for the last several years, anecdotally, I would say that it is generally a bit quicker to take the silver line. I will also say that despite this, I seldom (and only under specific circumstances) elect to take the silver line because I find it it is about as pleasant as punching myself in the face for half an hour.
However, there are two related factors which will erode if not reverse the minor time advantage the silver line seems to provide relative to the blue line (from South Station): What terminal are you going to/from and what time of day are you traveling . . .
The massport shuttle from the 4 terminals to the blue line stop is, more often than not (and always when it is busy) split so that passengers going to/from terminals A and B get on one bus and those going to/from terminals C and E get on another.
The nubbins silver line busses make all the stops at all the terminals all the time. When the airport is not crowded this has little adverse impact on how long it takes to get through. However, when the volume of vehicular traffic is heavy (roughly coniciding with rush hour on the highways during the work week) you can spend upwards of 20 minutes just dicking your way around the airport property fighting with the hotel shuttles, rental car shuttles, various other busses, cabs and privately-owned vehicles clogging the roads up. To speak nothing of the actual act of picking up and discharging passengers at each terminal--it is a show watching travelers (especially amateur travelers) lugging all their shit on and off the silver line when it is crowded, not to mention you can only board the silver line from the front-most single door by the driver (so you can pay--the massport busses are free so passengers can get on and off from both sets of doors, AND the busses themselves are smaller so it never seems to be quite as miserable an experience) . . . I kid you not, you can wait 5 minutes at a single stop just for people to get on and off the damned thing!
Personally, I'd rather get the exercise and walk from either Aquarium or State to South Station (on nice days when I am not in a particular rush, I'll even walk to the blue line station from the terminal) and avoid the soul-draining experience of a crowded silver line bus.
The only exception I typically make to this is if the weather is particularly rotten (very cold when I am not appropriately dressed to be outdoors for more than a minute at a time, or pouring rain).