Workshops/Survey Question Construction Using Qualtrics
_NOTOC__
Session Information
- Level - Intermediate, Conducting Survey Research Using Qualtrics workshop required
- Intended Audience - Designed for staff interested in learning more about constructing valid and reliable surveys, when to use certain question types, and how to avoid common mistakes when constructing survey items.
- Time - 50 minutes
Before the Session
This session is hands-on, so please bring your laptop if possible. If you cannot bring a laptop, one will be provided for you. Participants should be familiar with all of the material covered in the Conducting Survey Research Using Qualtrics.
General Description
Qualtrics makes it easy to distribute online surveys, but is your survey actually measuring what you need to know? Although there are a wide range of question types from which to choose in Qualtrics, the software doesn't write the questions for you. How do you construct a good survey question? Thoughtful survey construction practices can be the difference between meaningful data that supports decision-making and an uninterpretable mishmash of information. In this intermediate workshop, we will review some general concepts related to measurement, such as basic scale types, validity, and reliability. Good practices for survey construction will be shared and common pitfalls (e.g., double-barreled questions) will be demonstrated. Several Qualtrics question types will be explored for creating Likert, semantic differential, and other scales commonly used in surveys. Some frequently asked questions about survey construction will be covered, including survey length, number of response options, question ordering, and the use of forced-choice. The pros and cons of open-ended questions will be discussed. Finally, we will discuss issues related to the ethical treatment of respondents and policies related to the use of survey questions that generate sensitive, personal, and/or private data as well as questions that you may not need to ask because they have already been asked by others and the responses are available through IPAR.
Learning Outcomes
Following this session, you will be able to...
- Define the research questions underlying your prospective survey and the "constructs" that you are trying to measure
- Explain the difference between a theoretical and operational definition of a construct
- Explain the differences between direct and indirect measures
- Describe how the measurement of thoughts, feelings, and behaviors differs from the measurement of other variables (e.g., temperature)
- Provide at least one example of a nominal, ordinal, ratio, and interval scale and explain the differences among them
- Describe how the terms "validity" and "reliability" apply to survey research
- Provide at least one example of common survey item pitfalls: bias (e.g., leading, loaded), biased, double-barreled, irrelevant or inconsistent responses)
- Describe the basic steps involved in creating a Likert scale
- Create and edit all forms of multiple-choice and matrix Qualtrics item types
- Decide when to use constant sum, rank order, and side-by-side Qualtrics item types
- Determine when to use forced-choice items versus open-ended items or those with a "neutral" response option
- Choose between 5-choice and 7-choice Likert scale items
- Consider potential question order effects as source of measurement error in your survey
- Reorder questions and question blocks in Qualtrics to control for question order effects
- Evaluate the use of open-ended questions in your next survey
- Identify the new IPEDS categories for reporting race/ethnicity
- Explain how informed consent and debriefing are handled in your survey research project
- Determine whether any of your survey items gather sensitive, personal, or private data
- Submit your survey research proposal to the WSU Institutional Review Board for review
- Eliminate survey items based on an understanding of what institutional data already exists and can be applied to your project with the help of IPAR
- Reuse questions via libraries