|
|
| What is
a questionnaire?
A questionnaire
is a group of printed questions used to collect information from the people who
answer them (usually called respondents). The questions may be either
open-ended, where respondents are required to answer in their own words, or
multiple-choice, where respondents are required to select one or more answers from those provided. The respondents may also be provided with checklists or rating scales. The questions may be concerned with the
respondents' personal background, factual knowledge, attitudes or opinions,
for example, about capital punishment or changing from using the pound to the
euro.
(For more information on checklists, see the Checklists
topic.) |
 |
|
|
When
do you use a questionnaire?
|
|
When you are considering whether to use a questionnaire as a method of measurement or gathering of information, you should think about the following criteria:
-
Objectivity – how objective are
people's answers going to be? Will they just tell you what they think is the right or
'politically correct' answer?
-
Accuracy of measurement – will you be able to rely on the
results? For example, people tend to under-estimate their weight and over-estimate their height, so measuring these directly will be more accurate than asking people.
-
Validity – are the questions measuring what
you think they are? This can be quite hard to test but you need to be sure you are being properly understood and therefore that you are getting valid
answers.
-
Reliability – do people answer in a consistent way every time or will they just answer without really thinking about it?
-
Resource availability – what resources have you got to collect the
data?
|
|
How
do you get the right respondents? |
|
Getting the right people
- usually known as sampling - is an important consideration in
making sure that you eliminate any bias. If you can do this, you can scale up your results to the whole population you are interested in. All sampling should be random, as all the
common statistical tests used on questionnaire data depend on the sample being randomly selected. The bias can arise from three sources:
-
Sampling by non-random methods - may be due to personal prejudices of the person responsible for sampling,
for example, by excluding certain low income or racial
groups, or by just including your friends.
-
Sampling-frame bias - due to the
sampled population not accurately representing the whole population you are
looking at.
For example, you might select people alphabetically from a telephone directory, or
select students only, or select people in descending order of
income.
-
Bias through non-response from
those in the sample - your respondents may not
answer the questions because, for example, they cannot read or write.
|
|
How
do you collect the data? |
|
Although all questionnaires tend to be based on written questions, the responses to these questions may be collected in several different ways, including the following:
Personal interview (closed questions). These tend to be used for opinion polls and street information
collection.
Focused interview (open questions). These can gather in-depth information.
Telephone interview. Similar to
the first two methods, but tend to be shorter and are conducted over the telephone.
Panel. A group of volunteers complete questionnaires before and after
an event or intervention, to look for change.
Mail or postal survey. The questionnaire is sent through the post or emailed to people who may or may not have asked for them. The former are called
'solicited' questionnaires, and the latter 'unsolicited'. They can
give a very poor response rate.
Group interview or discussion.
An interviewer takes a group of people through a structured series of questions to find out about the range and depth of views and
opinions.
Self-administered questionnaires. These are usually presented to the respondent by an interviewer or a person in an official capacity, who explains the purpose but does not actually administer the questionnaire.
Group-administered questionnaires. These are similar to
self-administered questionnaires, but they are given to a whole group at the same time.
There is more
information about these methods in the Interviews
topic.
|
|
|
|
Stages in Questionnaire Design
There are a number of stages that you need to work through in questionnaire design.
|
Decide the aims of the study and what you are trying to investigate (usually called the
hypothesis or null hypothesis).
For example, men are more interested in sport than women.
|
|
Review the relevant literature, discuss with others and interested bodies. This also includes deciding how the data will be analysed and making sure that the data that are collected will be in a form that can be analysed.
|
|
Design the study, making the hypotheses specific to the situation.
For example, men are more interested in playing golf than women.
|
|
Choose the research method and way of collecting the information that best meets the needs of your study.
|
|
Carry out a small test (usually called a pilot study) to make sure you have chosen the best and most suitable research method and modify it if the results show this is necessary.
|
|
Select a sample of subjects to be approached.
|
|
Collect your data (usually called doing the field
work).
|
|
Sort out and process your data which includes giving codes to the answers so that a computer spreadsheet
can be used for analysis by one of the many statistical packages that are available.
|
|
Do the statistical analysis. |
|
Put all of the results together and see what they are telling you. This is the part where the hypothesis is tested to see, for example, if men statistically do play more golf than women. |
|
Write up the results, relating the findings to other research, drawing conclusions and interpretations. |
|
Problems of Questionnaire Design
The definition of the aims and the hypothesis of the work is one of the hardest steps. For instance, it is not enough to state that you want to investigate
'The attitudes of old people to their homes'. You need to define what attitudes you are interested in, what you mean by
'old people' and what sort of homes, how detailed the research will be and what topics will not be included. The logical sequence is first to decide what hypothesis is to be proved or disproved,
such as 'people over the age of 65 prefer to be able to stay in their own homes rather than move to sheltered accommodation', and what analysis and conclusions you would like to be able to draw.
Do you want to be able to differentiate by age, gender, disability, geographical area, etc.?
If you do, then you need to collect this information as well as the attitude data.
Secondly, you need to decide the statistical tests that you will carry out and what level of significance you are prepared to accept (e.g. 95% certain or 99% certain or 99.9% certain that you have got the right answer). This
has a bearing on the number of respondents you will have to find to allow your hypothesis to be proved correct (accepted) or incorrect (rejected). Next, you will need to work out the type of questions you will ask and how you will code and quantify them. This is particularly important with open questions, as you can only do this after you get the answers back
- you will not know the range of answers prior to this.
This is easier for multiple choice questions as you can code each of the options in advance. Finally, you will have to define your sample, how you are going to contact them and then actually find them and make contact.
|
|
|
|
|
Question sequence
|
The questionnaire will consist of a series of question sequences and you must first consider the order of the sequences. The layout may start with a set of factual questions, followed by questions about views or attitudes or be the other way round. Whatever arrangement, if spontaneous replies are required, the questions must avoid putting ideas or attitudes into the respondent's mind early in the questionnaire. It is also very important to make the questionnaire attractive and interesting to the respondent otherwise they may give up before the end.
It is better to start the questionnaire with some easy, impersonal questions and not ask for details like age, family, occupation and so on until later. Putting these questions at the beginning is unlikely to produce a helpful attitude. You also need to consider how the question sequence will strike the respondent. Is it too intimidating? Do the questions seem relevant to your explanation of the purpose of the research? Have you unintentionally made your own attitudes too obvious? Are the questions worded in a friendly way?
In the preliminary planning you may have lists of questions using several different techniques such as checklists, free-answer questions, and multiple-choice questions. You could decide to put all the checklist questions together at the beginning followed by all the free answer questions and ending with the multiple-choice questions.
|
|
This may ease the task of the respondent and give the questionnaire a more uniform appearance but the questions will no longer follow logically one after the other. The context of each question will be changed so the respondent will have the previous questions in mind when answering it, which may lead to incorrect answers. You must also consider the total amount of time and effort that you can expect a respondent to give. This is why it is necessary to carry out a pilot study on the final questionnaire with people who are representative of your respondents.
When using sets of questions where each set deals with a different subject, you must consider the phrasing of any particular questions in relation to its place in such a set of questions. People will tend to relate questions to each other even if you intended them to be separate. Conversely, the
'funnel' approach is a type of sequence where it starts off with a very broad question on a topic and then progressively narrows down the scope of the question until it comes to some very specific points. A filter question is used to exclude a respondent from a particular question sequence if those questions are irrelevant or not applicable to
them. The final choice of sequence must be determined by the individual survey and the results of pilot work.
|
|
|
 |
|
Question Wording
By the time you reach the stage of actually writing the questions, you should know specifically what is going to be asked. Question wording deals with how you ask for the information you need, without leading the person to a particular answer, or putting them off.
You should take care to ensure that:
-
The respondent is
motivated to respond. What will they get in return?
-
The respondent has the particular
knowledge required. There is no point asking people questions about how they feel about products that they do not
use, for example.
-
The questionnaire takes into account the
respondent's limitations and personal frames of reference, so they will easily understand the meaning and aims of the questions. There is no point asking people who have never been abroad about their attitudes to food in foreign countries.
-
The respondent has produced an
adequate answer. Open questions give more opportunities for full answers but respondents may be reluctant to spend time filling them in, so closed questions or multiple-choice boxes may actually bring out more information.
Broadly speaking, all questions are either 'closed' or 'open'. A closed question is one in which the respondent is offered a choice of alternative replies.
The advantages of closed questions are:
-
They clarify the alternatives for the respondent and avoid snap responses being given.
-
They reduce coding errors and time in analysis.
-
They eliminate the useless answer.
The disadvantages are:
-
It is difficult to make the alternatives mutually exclusive,
that is, all separate from each other with no overlap.
-
They must cover the total response range. (This presupposes that you will have a good idea of what you are likely to get, showing the importance of pilot studies).
-
They create a forced choice situation which rules out marginal or unexpected answers.
All the alternatives must seem equally logical or attractive.
-
In complex or difficult questions, respondents may dive for safety and the ease of the
'don't know' alternative.
Open or ' free answer' questions are not followed by any kind of choice, and the answers have to be recorded in full. In the case of a written questionnaire, the amount of space or the number of lines provided for the answer will help to determine the length and fullness of the responses you expect to
get. But remember that open questions are often easy to ask, difficult to answer and still more difficult to
analyse.
You must consider the following points when deciding the actual words to use in a question:
1. Is the question specific?
Often a common error is to ask a general question when an answer on a specific issue is wanted. The question,
'Are you satisfied with your canteen?' is unsatisfactory if you are interested in meal prices or the quality of the service.
2. Is the language easily understood by all?
Do not use technical words or words which have a different meaning for some people.
3. Is the meaning of the question clear?
It is important that the question is short. This ensures that questionnaire designers are clear in their thinking and remove surplus words. It ensures that the respondent is not overloaded with too much information and it reduces the chance of the respondent forgetting the earlier parts of the question. Double negatives and complex questions can lead to error as can the use of the words
'generally', 'normally' or
'frequently'. Avoid double-barrelled questions such as 'Do you suffer from headaches or stomach
pains?'.
4. Does the question give the respondent a lead to the answer?
Clearly, leading questions must be avoided. Obvious examples are 'Do you agree that the policies of the present Government are
unfair?' (which invites the answer 'yes'). More insidious are examples that use certain loaded words such as
'British', 'student' and the like, where the respondent is reacting to the word and not to the question. It is also necessary to be aware of questions which lead because of the nature of the questionnaire. An example is the question
'How many cigarettes do you smoke a day?' which may be innocuous in a questionnaire about household expenditure, but may produce different results in a questionnaire concerned with medical matters.
5. Will the question lead to prestige-biased answers?
Many factual questions are loaded with prestige. Some people will claim they take a bath or shower more frequently than is strictly true. People may claim to watch documentaries on the TV when in fact they are watching
'soaps'. People deny reading Sunday newspapers of dubious repute and claim to read more of the prestige papers than are actually
sold! Some people would seem to brush their teeth with great frequency and to visit museums almost every week! There is no simple answer to this problem. In addition to being aware
that it exists, there are two general measures that may help. Firstly, to use filter questions, or to word the main question in such a way that a low prestige answer is equally possible.
Therefore, instead of
'Have you read one of the following magazines within the last seven days?',
you could say 'Have you had time to read any magazines at all within the past seven days?'.
6. Is the question hypothetical?
These are 'What would you do if…?' type of questions. Generally these do not yield reliable results.
|
|
|
|
Piloting the
Questionnaire
This is a vital
stage. It is best done in three steps:
1
Individual criticism or feedback: the questionnaire should be handed to other people who preferably have some experience of questionnaires, for comment.
2
Depth interviewing: once the criticisms generated above have been corrected, the questionnaire should be given to a small sample of respondents (up to 10) for their reaction. On completion of the questionnaire, each respondent should be questioned in detail about the answers to the questions, to find out what the respondent understood the question to be asking, and the exact meaning of the responses given. Any changes should then be made to correct obvious problems.
3
Finally, the questionnaire should be given to a larger sample of respondents to investigate the implications of the analysis you wish to do and to check whether any invalid or meaningless patterns of answering are occurring. This also enables you to make estimates of the reliability and validity of the questionnaire. This stage should be repeated until the questionnaire appears to be error-free.
|
|
