Creating an effective online survey is a difficult proposition. Most people on the Internet are already saturated with information and requests to interact. Getting a sufficient number of quality responses requires some forethought. Here are ten things to keep in mind when constructing a web survey.
1. Know why you need a survey
Surveys offer a number of advantages—such as scalability, reuse, and flexible scheduling—but there are also drawbacks. Surveys tend to be biased toward those who self-select to take them, so getting a true random sampling is nearly impossible. They are usually anonymous with little oversight, which can lead to incorrect information being collected. Surveys often involve self-report, which means participants will respond in a way they perceive is right even if their actual behavior is quite different.
There are a lot of ways to get feedback. Not all of them involve a web form asking people to remotely answer a series of questions. Semi-structured interviews, focus groups, web site analytics, card sorting, and “crowdsourced” activities (such as open design or copywriting contests) are just a few of the alternatives to surveys. Make certain you are choosing to do a survey for the right reasons.
2. Respect your participants
The people who agree to fill out your survey probably have little obligation or investment in your work. Keep that in mind when writing invitations to participate and the wording for the questions you want to ask. Be courteous and grateful.
Respect should also extend to the data you keep. If you ask for anything that could identify an individual, let them know what you plan to do (or not do) with that information, as well as when you are going to get rid of it. Trust can be a barrier to participation. Being transparent can help lower that barrier.
3. Find the right question format
Questions come in many shapes and sizes. If you are looking for a single answer (such as an age group), don’t confuse matters by asking a multiple-choice question. Most online web form services will give you several kinds of formats for a given question. Try them out and see which one fits for the information you are interested in collecting. Similarly, standardized responses make it easier for the participant to understand the survey and give you useful data.
4. Keep it short
Because your survey is fighting for attention with web sites, videos, email, and a host of offline distractions, it is advisable to keep your surveys as short as possible … while still being useful to you. Don’t cut all of your questions down to a page for the sake of length if you need multiple pages to get the feedback you need. Trim the fat to streamline your questioning and take up the least amount of your participant’s time that you can.
Inevitably, some people will start your survey but not finish. This happens for many reasons, some of which you can’t control (i.e. network connectivity problems, family emergencies, etc.). Keeping this in mind, place your most important questions near the front of the survey, allowing you to collect as many of those answers as possible. Make use of “Save and Return Later” options, if they exist on your survey provider, but realize that the chance of someone actually coming back is low.
5. Organize to minimize confusion
Don’t cram all of your questions on to one page and call it a “short survey.” Long pages can be as intimidating as long surveys. It is much better to compose several small, simple pages with 1-2 related questions than to cut a 20-question survey into a few equal-sized chunks. Every new page requires some re-orientation, and the more coherent the content, the easier it is to get one’s bearings.
Measure length in terms of time spent. If you have more than one page on your survey, it is good practice to let people know how much more they have to do to reach the end. Progress bars or updated questions counts (i.e. “You have completed 9 of 20 questions.”) are a great way to give people an idea how much more time they have to spend to help you out.
6. Use conditional logic meaningfully
Conditional logic allows you to present (or hide) questions based on the responses your participant provides. At its best, it prevents your participants from answering questions that have no relevance. For example, you may want to ask how people use a particular social network in some detail. For those who don’t even have an account with that service, your questions could be meaningless. Putting a condition that automatically skips those participants to a later part of the survey—or asks a different series of questions to discover why they didn’t join—is a good use of conditional logic. If this doesn’t happen seamlessly, however, it puts some burden on the participant to navigate your survey in an odd way. Rewriting your survey questions or prescreening participants may be more effective.
7. Test your Survey
Be sure to put a few eyeballs on your initial survey draft to see if it makes sense. This can be done using a draft of the online survey, of course, but it might be quickest to use a text editor or sketch out the questions by hand. You want to find out if the wording you have chosen or the kinds of questions you are asking are confusing. Get feedback on the flow of the survey as well as its length. Make appropriate adjustments before sending it out into the world.
Keep in mind that changing your draft is easy. It has no consequence on any data because you haven’t collected any. The moment participants begin to contribute real data, any changes to the survey may invalidate all of the early submissions and force you to throw out data. It’s like carpentry: measure twice, cut once.
8. Timing is Important
Crafting a good survey takes some work, and you may be anxious to send it out quickly to start collecting responses. However, if you do that at 5pm on a Friday, odds are not good you will be happy with the results. Know when your target audience is most likely to be available and respond to your request. Make this decision based on participant input or other supporting evidence, whenever possible.
It is equally important to be able to follow-up on responses after the survey is released, to answer questions or help your network spread the word. If you aren’t available to respond, few people will want to invest the time to help you.
9. Incentivize through empowerment
For some kinds of surveys—particularly those that have to be long to get the data you need—payment is going to be a prerequisite to getting results. Paying people, whether with cash or discounts, can buy some obligation from your participants to complete the survey. Long-term, however, that can be a costly strategy with very little implicit incentive to participate.
Involving other people in your process gives them skin in your game. Advisory councils, focus groups, and development forums encourage people to become involved with your project, to the point they feel it is their project. They feel invested in the success of your survey and may take additional steps to help beyond simply completing the form.
10. Do Something
Surveys are part of a culture of ongoing customer relationships. Don’t treat your participants as disposable. They will want to get some ROI on the time they spent answering your questions. Whenever possible, share the outcomes of your data gathering with a report or blog, allowing your participants to learn something about themselves. Listen to what the survey results suggest, and apply the findings to some improvement. If you didn’t intend to do something with the information you got, why did you take the time to create a survey in the first place? (see #1 above)
Kevin Makice is a current Ph.D. student at the Indiana University School of Informatics and has authored a book, Twitter API: Up and Running. His research interests center around local use of technology and the application of relational psychology to complexity and design. Kevin is also the Director of Emerging Technologies for Hanapin Marketing, a Search Engine Marketing/Web Development firm based in Bloomington, Indiana.