Question

How do you avoid biased questions?

Answer 1

Use a team to develop the questions, and focus-group to test them.

Explanation:

We all have a perspective on anything we observe or do that can result in a bias in the way a question is formed. One internal check that may be helpful is to think about whether you think you already know the answer, or that 'most' people would answer it the way you would.

Questions should be as "outcome neutral" as possible, meaning that the way they are asked should not create a direction, or bias, towards a particular type of answer.

One example would be stating something in a particularly negative way, so that the (biased) reaction will tend to select a more positive (or opposite) response. You can see too many examples of this type of question in most "special interest group polls" that focus more on legitimizing their viewpoints than finding any real facts.

Using a team to formulate questions can reduce the possibility of a personal bias or communication format affecting the questions. Using a focus group to test the survey questions will help to identify ambiguous or unclear questions as well as potentially identify a bias towards one type of answer in some questions.

Answer 2

Think carefully about "what", "who", and "how". Be careful about multiple-choice questions.

Explanation:

1. Take the time to properly ascertain what you want to learn about, then frame your question accordingly. The question should come second; the intent, motivation or objective should come first in your train of thought.

2. Choose the respondents carefully, and ensure that they are the relevant and appropriate people for answering the question. Have you excluded suitable respondents, or included unsuitable respondents?

3. The method of collecting responses must reach all possible respondents, not just the ones whom you can reach conveniently. In more statistical terms, avoid convenience sampling.

4. If the question involves multiple choices, ensure that all options are available; don't restrict the options to the ones that you favour.

5. Randomize the order of your questions and, if needed, the order of the choices in a multiple-choice question.