You don't really turn down someone who's invited you to do something. What you do is say, "I'll do my best," and then not show up. Last week almost everyone in the company went to an island where no expats had been before, but I declined to go: I still had lesson plans to write, and I was simply exhausted. Because we had to sign up before the trip, I couldn't pull the whole "I'll do my best" trick. A lot of the staff got butt hurt I didn't go: they took it as a sign that I wanted to be far away from them. In this high context, it's hard to be alone.