Bias (Part 1)

When I say "bias," I'm not talking about personal bias.  I'm talking about selection bias.  It's the bane of statisticians all over the world.  It gets into your data and makes all the results suspect.  It's notoriously hard to prevent.  Let's say we want to do some research on domestic violence.  We want to recruit a large representative sample of the population.  Well, one of the most infamous forms of bias is volunteer response bias.  Those who have a bone to pick will be far more willing to be part of the sample than others.  If we allow them to, they will hijack the data and make all the results meaningless.  Yet for so many studies, even those published in large peer-reviewed journals, this kind of thing often happens.  In my example, if I were trying to recruit a large sample, those who had very strong feelings about domestic violence would be the first to turn in surveys and share it with their friends.  Even if my forums for disseminating the questionnaire were unbiased and universally available, this bias upends my results.  However, if my main forums for disseminating the survey are primarily viewed by those with interest in domestic violence issues, the bias compounds.  Would my resulting data be useful?  Sure.  But it would not be a good representative sample of the whole population, and I would have to take that into account when interpreting the results.

I have reason to believe that there are a large number of Mormon men who are attracted to other men, who also have been sealed to wives to whom they are committed.  They do not identify as gay, nor do they frequent forums where these issues are discussed.  They have told nobody about their orientations, sometimes not even their wives.  Why would they need to bring it up?  They might feel threatened by the gay rights movement and so largely ignore anything to do with it.  They might only rank a 4 or 5 on the Kinsey scale.  These people are a very important demographic and we have absolutely no idea how big it is.

Some people will say that this group can't be very big, because few people could pull that off.  To this crowd, I say "Wake up and smell the coffee!"  I've met people who feel the exact same way about coffee.  There's no way a whole group of people could eschew morning Java and still function normally.  Or teen sex.  It's naive, they think, that a church teen program could actually expect to convince a majority of its members to wait until marriage before sex.  Sorry, but despite the doubts, it is currently happening within the Mormon church.  So I think it's possible that the silent gay-oriented Mormon population could be very much larger than expected.

So what of this recent study on gay Mormons and marriage?  It's useful as a research tool, but I think it's naive to infer results about the general population using the study.  There's way too much chance for selection bias.