Workplace Like a Family?

For the past eighteen months I've been working for a very small company. The owner has this idealized notion of us as a kind of "family."

What that means is, apart from working really hard, we're expected to take part in a lot of after-hours team building-type stuff. Bar hopping. Dinners. Activities.

Participation isn't mandatory, but, like I said, it's a small organization and you can't really get out of doing things in a graceful way. Not participating tends to color the owner's opinion of you. You're not a team player. You're not one of the gang. Or, to put it another way, you're not part of the family. It's extremely cliquey.

I've seen people get fired for different reasons. Inevitably, though, they're the ones who didn't want to join in the fun. They don't get fired because of not wanting to attend a party or suchlike - but on some level it plays a part in their termination. How can it not?

Lately I find myself resenting the situation more and more. It just eats into my home life too much. I'm not sure what to do because, otherwise, it's a pretty good gig. And considering the state of the economy, I know I'm lucky to have anything at all.

Do any of you have to deal with something like this? How common is it? Is it just a small company thing? If I don't enjoy it, am I better off just leaving or do I put up with it?

What say you?