Would you take it personally? How would it affect your working relationship?
ETA: I'm the boss who always says no. I feel bad about it, and because every "no" is followed with a "why," I feel like I need to have an excuse other than "I just don't want to and also think it is kind of inappropriate." Thanks all for bolstering my confidence. I think the minion and I will have a talk tomorrow.