Surveys
✅

Surveys

Tag(s)
Research & testing
Cook(s)
Rebecca BlakistonRachel Brown

Nutrition profile

Use surveys to collect input directly from respondents, such as their attitudes, opinions, and self-reported behaviors.

Cooking time

Preparation: ~5 minutes per question

Data collection: 1-20 minutes per participant

Analysis: 1-30 minutes per question

You can create and conduct lightweight, simple surveys in a short amount of time (a few hours or days). More in-depth surveys will take longer to develop. Expect the analysis of the results to take the longest amount of time.

Perfect for

Learning what lots of people think about a topic. You can gather preferences, opinions, reactions, and stories from a large amount of participants in a short amount of time. Surveys are generally web-based, remote, and unmoderated, meaning respondents can complete them on their own time.

While surveys can be used to gather quantitative or qualitative data, they tend to work best for quantitative data. This is because you can gather "big" but not "thick" data.

Prep work

Determine the purpose of the survey. Surveys are best at telling you what people think and say (attitudinal data) rather than what people actually do (behavioral data). But you can collect self-reported behaviors or anticipated behaviors, too, as quantitative behavioral data might be difficult to obtain with other methods. For example, you might survey people to ask if they attended an exhibit in the past year or if they plan to travel over spring break.

Clearly articulate your research question(s) to validate that a survey is the best tool to answer that question. What are you trying to learn? Who are you trying to learn from?

Each survey should have a clear, singular focus. And you should have a plan for how you will analyze and act upon the data from each and every question you are asking.

Good examples of research questions:

  • Do students find option A or option B more appealing?
  • What do employees like about X?
  • How do faculty prioritize this list of items?
  • What percentage of off-campus students regularly eat on campus?

Ingredients

While paper surveys are an option, we recommend using a digital survey for larger reach and easier analysis. So you'll need a digital survey tool, such as:

Most survey tools have data visualizations built in, but you might benefit from another tool to export, analyze, and visualize the data, such as:

Directions

Writing survey questions

Keep your survey questions short, friendly, specific, and unambiguous. Treat your survey as a brief conversation with your respondent, and include as few questions as possible.

Don't make answers required unless absolutely necessary for your analysis. Required questions force respondents to give you bad data rather than just skipping a question they don't understand or that isn't relevant to them.

Writing multiple choice questions

Multiple choice questions are often your best option since they are quick and easy for participants to answer. Use radio buttons when respondents can only select one answer, and check boxes when they can check all that apply.

Make sure you:

  • Include all possible answers; if impossible, include an "Other" option.
  • For most questions, randomize the order to avoid bias (except for "Other," "Prefer not to say," and/or "Not applicable" responses, which you should anchor to the bottom). For identity questions, use static alphabetic (e.g. Asian or Asian American, Black or African American) or numerical order (e.g. 0-24, 25-35). Avoid suggesting a hierarchy between identities of people.
  • Use parallel grammatical structure for your choices (e.g. all nouns, all gerunds)
  • When using number ranges, such as for ages or date ranges, make sure options don't overlap (e.g. 0-10 and 11-20 rather than 0-10 and 10-20)
Example of a multiple choice question with checkboxes (check all that apply) in Google Forms
Example of a multiple choice question with checkboxes (check all that apply) in Google Forms
Example of a multiple choice question with radio buttons in Google Forms
Example of a multiple choice question with radio buttons in Google Forms

Writing Likert scale questions

Likert scales are rating scales that are good for measuring satisfaction or agreement, and comparing preferences or reactions to options or components of something. The scale allows more detailed responses than just "yes/no." Likert scale questions can help you measure satisfaction levels and can be helpful for comparisons over time.

When using a Likert scale:

  • Use an odd-numbered scale of 3, 5, or 7 options. 5 is usually sufficient, though 7 can provide more nuance. Don't go higher than 7, since it causes increased cognitive load on respondents without providing additional value to the study.
  • Order options from negative or lowest to positive or highest.
  • Provide an equal number of positive and negative options (a full range of responses), and always provide a neutral option in the middle. Indifference and uncertainty is still informative, and you want to give respondents a neutral option to avoid them giving you bad data.
  • Label the options with words rather than numbers.
  • Frame the question as a question over a statement.
  • Avoid bipolar scale options. Use the same word throughout unless things are clearly opposites. Some things are clearly opposites, such as "bad" and "good," but other descriptors are less clear. For example, "safe" might not be the opposite of "risky," so "unsafe" would be better.
Example of a Likert scale question in Qualtrics
Example of a Likert scale question in Qualtrics

Writing open-ended questions

Open-ended questions give respondents an empty text box and ask them to answer in their own words. These responses can provide helpful qualitative data, including quotes and stories.

Minimize use of open-ended questions since they take longer for respondents to fill out and longer for you to analyze. Many respondents will be using a mobile device, which can also make them more difficult to respond do.

If you find yourself writing a lot of open-ended questions, a survey might not be the best research method. User interviews might be a better choice.

That said, a helpful open-ended question at the end of many surveys is often a simple: "Tell us more."

Example of an open-ended question in Google Forms
Example of an open-ended question in Google Forms

Plating

Introducing the survey

Title the survey something meaningful, starting with a verb. Good survey titles:

  • Give us feedback on X
  • Vote on X
  • Help us X
  • Submit your ideas about X

Some introductory content is fine to let respondents know what the survey is about, but avoid overwhelming them. Too many surveys have paragraphs of instructions before the first question. Unless required for legal reasons, such as informed consent, keep it as short as possible.

Example brief introductory text for a Google Form survey
Example brief introductory text for a Google Form survey

Structuring the survey

Some of the best surveys are on a single page and 5 questions or less. But if you have more than 5 questions, consider splitting them into multiple sections or pages and adding meaningful titles or headings to allow for skimming.

When you have multiple pages, progress bars can help motivate respondents and increase completion rates. Progress bars are an option included in most online survey tools. Display your progress bar at the bottom of each page and keep it simple.

Example progress bar in Qualtrics
Example progress bar in Qualtrics

Closing out the survey

Thank your participants once they complete the survey. Customize the confirmation message to remind respondents how you plan to use the data. You can also include contact information (when possible) and information about next steps if any.

Example confirmation message
Example confirmation message

Pro tips

  • Beware of survey fatigue. Keep surveys to 10 minutes or less to fill out and make every question count.
  • Assume users will take a survey from their mobile device in portrait mode, so test how your questions appear. For Likert scales, for example, you might need a vertical layout for it to work on mobile. Grid layouts can be especially problematic on mobile.
  • Keep it neutral. Don't ask leading questions. For example, "Describe how you use the information shared with you over email" is more neutral than "How helpful is the information shared with you over email?"
  • Don't collect personal information if not absolutely necessary, as you might lose people. If you need demographic information, make these questions inclusive. Allow multi-select and include opt-out options like "Prefer not to answer" and "Does not apply." For identity questions such as race, ethnicity, or gender, include "Another option not listed (please specify)" to avoid othering people who identify outside normative categories.
  • Pre-test the survey and get feedback before you launch. It will help ensure your questions are unambiguous and there aren't any usability issues.

Resources

How to Write Survey Questions (That Collect Better Data) by KwikSurveys

How to Write Great Survey Questions (And Avoid Common Mistakes) by Sarah Fisher and Scott Smith

8 Types of Survey Questions to Get You All the Data You Need by Nemanja Jovanc

Big Design 2017 Presentation on Survey Design by Lauren Isaacson

Four Strategies for Crafting Inclusive and Effective Demographic Questions by Jennifer Frederick

Credits

Why Big Data Needs Thick Data by Tricia Wang

Tips for Making Better Forms by Rebecca Blakiston