Introduction. Cognitive Process and Question Types
- Four steps: interpretation, information, judgment, response
- Optimizing behavior: self-expression, self-understanding, altruism, challenge
- Weak and strong satisficing: arbitral, irrelevant
- Satisficing factors: task difficulty, respondent ability, motivation
- Open questions: rich in meaning, time-consuming
- Closed questions: comprehensive, less reliable
Rating Scale and Its Length
- Scale length affects validity, reliability, clarity, satisficing
- Dichotomous scale: clear in meaning and inconsistent
- Trichotomous scale: clear and not accurate
- Long scales (101-points): incomprehensible, prone to errors
- Midpoints lower motivation, improve reliability
- “7-point scales are optimal in many instances” Krosnick & Presser (2009, p. 283)
Labeling and Order Response
- Verbal labeling increases respondent’s satisfaction and reliability
- “Yes/no” questions: practical, promote “acquiescence” Krosnick & Presser (2009, p. 21)
- Acquiescence reasons: fatigue, politeness, personal characteristics
- Possible approaches: adjust/exclude answers, “item reversal”
- Primacy effects: attributable to “weak satisficing”, verbal
- Recency effects: attributable to short-memory, oral
No-Opinion Answers
- +“Don’t know” answers help not to appear uninformed
- + the lack of necessary information and experience
- +/-Help to escape socially undesirable images
- -DK options prevent generation of meaningful answers
- -Do not increase reliability and validity of the survey
- -Encourage satisficing by providing shortcuts
Response Bias
Reasons: favorable image, reduce punishment, increase rewards
Possible spheres: racial prejudice, political views
Ways of counteraction:
- “Bogus pipeline technique”
- “Randomized response”
- “Item count method” Krosnick and Presser (2009, p. 301)
- Anonymity: self-administered questionnaire, web surveys
Error Sources
Main sources: “comprehension” and “frailties of memory” Krosnick and Presser (2009, p. 305)
Possible ways to avoid errors:
- Interviewer’s instructions to the respondent
- Long, redundant phrases and sentences
- Explicit instructions about the importance of survey
- Simplified tasks and wording of items, decomposition
Question Order
- The main aim is to minimize errors
- Question order major types: serial, semantic
- Serial order affects: motivation, fatigue, learning
- Semantic order affects: coherency, logic
- “Funnel” order (general and specific items) Krosnick and Presser (2009, p. 313)
- Reasons for specific order: context, survey focus
Questionnaires Evaluation
Methods without data collection:
- Expert review (critiques of the questionnaire)
- The Questionnaire Appraisal System
- The Question Understanding Aid
Methods with data collection:
- Behavior coding
- Response latency
Reference
Krosnick J.A., & Presser S. (2009). Question and questionnaire design. In J.D. Wright & P.V. Marsden (Eds.), Handbook of survey research (pp. 263-314). Bingley, UK: Emerald Group Publishing.