One of the most frequent tasks the staff in Student Life Studies do is helping people design good survey questions. It’s a science and an art, it takes practice and an eye for detail. As with many aspects of assessment, you have to have a clear purpose statement in order to craft questions that help you get to what you need to know. In addition, the types of questions you use are dependent on what you want to know.
I’m going to highlight several different questions types common in survey design. They each have their uses, advantages, and disadvantages. If you would like more information, see Student Affairs Assessment: Theory to Practice by Henning and Roberts.
The simplest question type is yes/no. That means a condition exists or it doesn’t. Sounds simple, but there may be times when the respondent isn’t sure or doesn’t know the answer, and you have to determine if you need that third option. Sample yes/no questions are: Are you at least 18 years of age? Did you attend the alcohol education workshop on February 1, 2020? Are you registered to vote in Brazos County?
Another type of question is choose one. In this case, you want respondents to pick one answer from a list or series. For example, you could ask respondents to specify which state they currently live in. There would be a list of 50 states (and the District of Columbia) and possibly an “I don’t live in the United States.” It should be fairly easy for the respondent to answer the question. You could also be asking about days of the week, months of the year, how people heard about a particular program, their favorite session of a conference, what residence hall they live in, etc. The order of the responses should make some sort of sense: alphabetical, chronological, etc. Just think, you would not want to see a list 50+ states in a random order where you had to search for your state. Usually, you want to keep the list fairly manageable, so respondents don’t get lost in a long list of items.
A choose all that apply question also provides a list of items, but the respondent has the option to pick more than one answer. When you create the response options, you want to keep them to a reasonable number in the list, just as you would for a choose one. You also want to specify the maximum number that respondents should choose, if you have that limit. The instructions could be worded as “choose up to three responses” for example. A common question is something like, “How did you hear about the program?” The answers could be a list such as flier, newspaper, radio, social media, word of mouth, and other (with a write in option). A choose one question and a choose all that apply question could be very similar in wording.
A very common type of question is a rating scale. You have probably seen these at restaurants, online retailers, or workshop evaluations that are seeking a satisfaction score. The questions ask respondents to indicate a strength of response. Common types of scales are strongly agree to strongly disagree, always to never, excellent to poor, etc. The scales are usually three to seven responses in length. People ask whether there should be a “neutral” middle option. I believe people can be neutral in their opinion, but it’s also okay if you don’t want people to be fence-sitters and remove the middle option.
Similar to ratings, but without the granularity, is the ranking question. In this case, the respondent is given several statements or list items to put in a particular order (most important to least important, most frequent to least frequent, etc.). When you get the results back, you will know the order of how people ranked the items, but not the closeness or weight of the answers. Think about a horse race. The distance between the first and second horse doesn’t matter; a horse could win by a nose or by a length. If you wanted to know the strength of opinion, then you could convert your ranking into a rating scale by including a scale for each one of your ranking items. With ranking questions, you should be clear about how many items you want ranked: all of them, the top five, etc.
Open-ended questions allow respondents to use their own words to answer. When you are creating those questions, be sure they do not fit into another type of question category. For example, asking, “Did you learn something from the program?” which could be answered yes or no, is different than asking, “What did you learn from the program?” While the other questions types are quantitative and possibly easier to analyze, the qualitative questions will give you more descriptive information, but may be more time-consuming to analyze.
That is a quick overview of question types. If you would like assistance with designing good survey questions, please reach out to Student Life Studies. We are here to help.