How to Design Surveys
Aug. 27th, 2009 10:27 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
prkResearch Seminar at ECU
Purposes of Survey Design:
1: Break it down into manageable questions people can answer.
2: Encourage, motivate and uplift people to get them to participate.
3: Reduce response errors.
Incentives to respondents can assist in motivation.
You can't just ask questions and try and interpret the results. You need
to understand the questions you're asking.
Eg questionnaire for JJJ Hotest 100 didn't include questions on
respondent demographics. More respondents than listeners. Started
analysing responses & songs, but not respondents.
Have to have a very good understanding of research questions, target
group and what you want to do with the information - how to analyse,
etc.
Need to think and plan out everything ahead.
Steps in Survey Design:
Develop clear objectives.
Develop research methods (ethics, audience, data collection, questions, etc).
Types of Survey:
- Self administered
Paper
Internet
- Interview administered
Face to face
Telephone
Analysis and presentation - you have to know how to analyse the data.
Collecting specific types of data will only allow specific analyses and
thus only specific outcomes.
Must learn how to engage the target audience.
Curiousity is the luxury of the whole field.
If you want to know something - ask it. Don't bat around the bush if you
are able to directly ask the question.
Very specific instructions for every single question, else you don't
know if people answered it the way you wanted.
Validity - how you know whether the questions you ask match your
research objectives. Any testing around whether the questions being
asked.
Reliability - is the measure stable over a period of time. Scale which
is out will give reliable but not valid measurements.
The best surveys have:
Specific objectives
Straight forward questions
Sound research design
Sound choice of population or sampler
Reliable and valid survey instruments
Appropriate management and analysis
Accurate reporting of survey results
Reasonable resources
Open ended questions
- Allows opions to be expressed
- provides greater information
Closed questions
- Concise
- Must allow "Don't know" or "I refuse" options for coding, even if not
shown to respondent.
- Make code frames complete before survey begins!
Probing
- Get respondents to enlarge on what they've said
- Clarify what respondents have said
- Explain the reason behind what they have said
Rating & Ranking scales
- Rotate order to overcome bias / avoid respondent becoming bored
- May require probing
Demographic questions
- Ensure you have information about the respondents
- Gender, Age, Household structure, Education level, Income, Postcode, etc
Use transition statements in the questionnaire
- Maintains interest of respondent
- Re-establishes direction for the interview
Pilot the survey!
- Ensure flow
- Ensure questions are answered in the correct order
- Discovers any flaws
- Test the questionnaire for length and repetition
Reassure the respondent - who you are, why it's being done, not selling
anything, etc
Wrap up with a post-amble - gives a sense of completion for the
respondent & reassures them. Thank them.
Purposes of Survey Design:
1: Break it down into manageable questions people can answer.
2: Encourage, motivate and uplift people to get them to participate.
3: Reduce response errors.
Incentives to respondents can assist in motivation.
You can't just ask questions and try and interpret the results. You need
to understand the questions you're asking.
Eg questionnaire for JJJ Hotest 100 didn't include questions on
respondent demographics. More respondents than listeners. Started
analysing responses & songs, but not respondents.
Have to have a very good understanding of research questions, target
group and what you want to do with the information - how to analyse,
etc.
Need to think and plan out everything ahead.
Steps in Survey Design:
Develop clear objectives.
Develop research methods (ethics, audience, data collection, questions, etc).
Types of Survey:
- Self administered
Paper
Internet
- Interview administered
Face to face
Telephone
Analysis and presentation - you have to know how to analyse the data.
Collecting specific types of data will only allow specific analyses and
thus only specific outcomes.
Must learn how to engage the target audience.
Curiousity is the luxury of the whole field.
If you want to know something - ask it. Don't bat around the bush if you
are able to directly ask the question.
Very specific instructions for every single question, else you don't
know if people answered it the way you wanted.
Validity - how you know whether the questions you ask match your
research objectives. Any testing around whether the questions being
asked.
Reliability - is the measure stable over a period of time. Scale which
is out will give reliable but not valid measurements.
The best surveys have:
Specific objectives
Straight forward questions
Sound research design
Sound choice of population or sampler
Reliable and valid survey instruments
Appropriate management and analysis
Accurate reporting of survey results
Reasonable resources
Open ended questions
- Allows opions to be expressed
- provides greater information
Closed questions
- Concise
- Must allow "Don't know" or "I refuse" options for coding, even if not
shown to respondent.
- Make code frames complete before survey begins!
Probing
- Get respondents to enlarge on what they've said
- Clarify what respondents have said
- Explain the reason behind what they have said
Rating & Ranking scales
- Rotate order to overcome bias / avoid respondent becoming bored
- May require probing
Demographic questions
- Ensure you have information about the respondents
- Gender, Age, Household structure, Education level, Income, Postcode, etc
Use transition statements in the questionnaire
- Maintains interest of respondent
- Re-establishes direction for the interview
Pilot the survey!
- Ensure flow
- Ensure questions are answered in the correct order
- Discovers any flaws
- Test the questionnaire for length and repetition
Reassure the respondent - who you are, why it's being done, not selling
anything, etc
Wrap up with a post-amble - gives a sense of completion for the
respondent & reassures them. Thank them.
