Bias (Part 2)

Last time I talked about some of the problems of selection bias in statistics, in particular in studying gay Mormons.  So how can the effects of this bias be lessened?  I think there are several ways, but they are difficult to pull off.

There needs to be a way to get a cross section of the population.  Let's suppose our population of interest is the adult men in the church.  One possibility might be to make a random selection of wards within the population of interest (say, members living in the US) rather than recruiting people online or other such highly biased method.  Anonymous surveys passed out in Elders/High Priest quorums might be appropriate, but care should be taken.  Make sure that the questions on the survey are worded to encourage the kinds of honest answers that are relevant.  For example, "I consider myself to be (a) gay (b) bisexual (c) heterosexual (d) same-sex attracted (e) other" might not be a good question, since many men might interpret their identities as heterosexual, despite being a 4 or 5 on the Kinsey scale.  A better way might be a survey that first assures them of their anonymity, explains the purpose for the survey is for a trusted organization (like, perhaps, LDS social services) to gain accurate information in helping to guide the youth of the church, and asks them if they have ever found themselves attracted to members of the same gender, regardless of the way they define their orientation.  There's probably even a better way to word the questions.  I'm just shooting from the hip here.  But the wording of the question has to assess attraction, not identity, if it is to be a useful measure for answering the kinds of questions for which we want answers.

This kind of survey would require a lot of trust from church leaders, possibly working with LDS social services or some other entity within the church.  But without this kind of data, I'm afraid the results are not very strong.  They may have merit for the portion of the population that is likely to respond, but still fail to generalize to the church as a whole.

Getting good data is always a difficult task, and data about a cultural taboo (sexuality is often a taboo subject within Mormon culture) is particularly hard.  But until we have good data, we have to regard any results with an appropriate level of skepticism.  Too many times throughout history, faulty data has led to some very poor decision making because of selection bias.