Personally, I never cared if he built the entire thing himself, but that's a hugely great PR move. He's taken one of the anti crowd's favorite points and cut it off at the knees.
I expect he is taking a very narrow view on this recusal. How many hotels and other facilities will be built in the next ten years on the backs of these olympics that will not house athletes or events? Don't worry about Suffolk. They will make plenty of money off this. His declaration is pretty hollow.