How to write survey questions that don't steer the answer
A badly worded question does not just get fewer answers. It gets confident, wrong ones that look exactly like good data in a chart. The four defects below cause most of it, and all four are fixable before you publish.
Two tests every question has to pass
First: could two reasonable people read this differently? "Do you use the product regularly?" is answered honestly as yes by someone who opens it daily and by someone who opens it once a quarter, because regularly is doing all the work and nobody agrees on what it means.
Second: does the wording tell the respondent which answer is expected? If a question contains an adjective you would use in your own marketing, it is leading. People are cooperative — most of them will nudge toward the answer they think you want, particularly when they know the company is reading.
These matter more in a survey than in conversation because nobody can ask you what you meant. The respondent answers the question they think you asked, you get a number, and afterwards there is no way to tell which reading produced it.
The four defects that ruin most surveys
Leading questions
"How helpful was our support team?" presupposes helpfulness and offers only degrees of it, so the person who found support useless has to argue with your premise before answering. The same trick hides in preambles: "Given the improvements we've made this year, how satisfied are you..." has told the reader the answer before the question arrives. Strip the adjectives and let the scale carry the judgement.
Double-barrelled questions
"Was our response fast and accurate?" has no valid answer for a fast wrong reply, which is the exact case you most need to know about. The test is mechanical: if you can split the question on "and" or "or" into two questions that could get different answers, it is two questions and should be asked as two.
Unbalanced scales
Excellent / Very good / Good / Fair has three positive points and one soft negative. Whatever the average comes out as, it is a property of your scale rather than of your customers. A balanced scale has the same number of positive and negative points around a defined middle, and uses matching intensity words on both sides — very dissatisfied opposite very satisfied, not opposite delighted.
Assumed premises and broken option lists
"Which of our features do you use most?" assumes they use one. Option lists need to be exhaustive and mutually exclusive: give a "none of these" escape, and never let ranges overlap (0–5 and 5–10 puts the person who answers five in both buckets, and they will pick whichever they read first).
Six questions, and what to ask instead
Each rewrite removes the frame and lets the answer come from the respondent rather than the question.
| The question as written | What it does to your data | A version you can trust |
|---|---|---|
| How much did you enjoy the new dashboard? | Presupposes enjoyment; people who disliked it must fight the frame to say so | How would you rate the new dashboard? (Very poor to Very good) |
| Was our support fast and helpful? | One answer for two attributes, so a fast wrong answer has no honest response | Split it: rate the speed, then rate the helpfulness |
| Do you agree that checkout is easy to use? | Agreement wording invites acquiescence — saying yes is the path of least resistance | How easy or difficult was checkout? (Very difficult to Very easy) |
| How often do you use the app? (Rarely / Sometimes / Often) | Every respondent defines those words differently, so the buckets aren't comparable | In the last 7 days, on how many days did you open the app? |
| What do you dislike about the pricing? | Assumes dislike and manufactures complaints from people who had none | Ask how they feel about the pricing first, then follow up on the negative answers |
| Rate us: Excellent / Very good / Good / Fair | Three positive points and no real negative; the average is a property of the scale | Very dissatisfied / Dissatisfied / Neither / Satisfied / Very satisfied |
Where a rewrite splits one question into two, cut a question elsewhere rather than growing the survey.
Order changes answers, and there is no neutral order
Ask about a specific irritation and then ask for an overall rating, and the overall rating will move toward the irritation — it is the thing you just put in the reader's head. This is why general questions belong before specific ones on the same topic. Asked first, the overall rating reflects what the person walked in thinking rather than what your last question reminded them of.
People also try not to contradict themselves. "Would you recommend us?" asked immediately after someone has listed three complaints gets a different answer than the same question asked first, because the earlier answers are now a position they feel they should be consistent with. Neither ordering is objectively right; what matters is that you pick one and keep it identical across waves, or your trend line is measuring your questionnaire.
Keep every scale in the survey running the same direction — negative on the left, positive on the right, or the reverse, but only one. Flipping direction mid-survey to catch inattentive respondents mostly generates noise from attentive ones who were reading quickly by then.
Put demographics and identifying questions at the end. They are the questions people abandon on, and by the time they appear you already have the answers that mattered. The exception is a screener that decides whether someone should take the survey at all — ask that first, and let conditional logic route the people who do not qualify to an ending rather than making them answer questions you will discard.
For long option lists, randomize the order if your tool supports it. Options near the top of a list get chosen more often simply because they are read first, and a satisfied-enough respondent stops reading as soon as something fits. If you cannot randomize, at least avoid ordering options with your preferred answer at the top.
The review pass
Fifteen minutes with this list catches almost everything. Do it after the survey is written, not while writing.
- Read every question aloud. If it needs a second breath, it is probably two questions.
- Search the question text for "and" and "or" in the stem.
- Count the positive and negative points on every scale. They should match.
- Replace vague frequency words with a time window and a number.
- Check every option list for an escape hatch and for overlapping ranges.
- Confirm all scales run in the same direction, top to bottom of the survey.
- Move demographics to the end, and screeners to the start.
- Have someone outside the project take it and tell you, in their words, what each question was asking.
Questions people ask
What is a double-barrelled question?
One that asks about two things but allows a single answer, like "Was the service quick and friendly?" Respondents answer whichever half is stronger for them, and you cannot tell which half you measured. Split it into two questions.
Should I include an "other, please specify" option?
Yes, whenever your list might not be exhaustive. It costs you a tidy chart and some manual reading, and it buys you the answers you did not think to offer — which is often the most useful thing in the whole survey.
Is it okay to make survey questions required?
It is fine for questions every respondent can genuinely answer. It becomes a data-quality problem the moment someone has no valid answer, because they will pick something arbitrary to move on. Add a "not applicable" option or leave the question optional.
How many questions should a survey have?
As many as there are decisions you will actually make with the answers. Length is a cost paid by every respondent, so each question should be earning its place — and cutting the ones that are not is the cheapest improvement available to you.
Should I use open-ended questions?
One well-placed open question at the end is usually worth more than three in the middle. They are expensive to answer and expensive to analyze, so use them to catch what your closed questions missed rather than as a substitute for deciding what to ask.
Keep reading
Better forms.
Better data.
Build your first form in under 60 seconds. Free forever for personal use, no credit card required.