Online surveys imposed themselves as the most efficient way and the
fastest way to collect data with each kind of public: consumers,
customers, collaborators…
But with the request multiplication to answer questionnaires on
Internet, we assist to the same phenomenon than the one that happened
for the surveys by phone: a regular increase and constant refusal and
abandon rates during a questionnaire, and the most troubling, the random
answers or voluntarily wrong answers rate.
The explanation of these phenomena is essentially linked to the
structure and to the management mode of web surveys. The challenge for
all online questionnaires authors is to adopt the most efficient
techniques in order to optimize the answer rate and the quality of the
collected answers.
Here are ten advices coming from the best experts in order to optimize your online surveys…
Rule n° 1 : Shorten your questionnaires
First rule consists in doing a short questionnaire. This can seem so
obvious when we look at the interviewed people side and when we think
about these endless questionnaires that we often get and rarely
finish.
But when we are in the other side and that we have to design a
questionnaire, big is the temptation to talk about everything by
multiplying questions, precision demands and precision comments. Some
surveys responsible persons are only interested in their aim and their
problem and they forget the fact that they face people and not machines
to answer their questions. Those who pay the respondent people sometimes
tend to consider the payment or the bonus as a way to give them the
right to ask anything to people and to provide long questionnaires. In
this case, respondent people will perhaps fill the whole questionnaire
but with the high temptation to give bad quality answers.
A survey from the American organism AFR (Advertising Research
Foundation) perfectly showed it by having established a long
questionnaire that clearly was counterproductive and even risky. This
association was mandated by 17 huge surveys institutes and by many big
companies (Unilever, Coca-Cola…) in order to establish the FoQ
(Foundation of Quality) in the online surveys field. Its survey, that
had a $1 million budget, proved that a 30 minutes questionnaires
multiplies the risk to obtain bad quality answers per 6 compared to a
questionnaire about the same theme but expected to only last 15 minutes.
Our advice goes further: limit the time of your questionnaires and make
them last only few minutes (5 or 10 maximum) by concentrating on only
one clear issue and by resisting the converse temptations and resisting
the demands of the ordering persons. Whether the number of researched
elements does not enable to do a short questionnaire, the solution is to
divide the questionnaire in several shorter questionnaires. Give clear
and exact information to respondent people about the real time of the
questionnaire. Also use methods and advices from the next slides in
order to give to the respondent the feeling of having a short
questionnaire (conditional displays and pages, compact layout, cursors
utilization…).
Rule n° 2: Take care of the appearance
Today, internet users are used to have a visual ergonomy and quality
level to respect in the questionnaires made for them.
Of course, it is not about making a too sophisticated and too original
questionnaire with special effects, animation or too artistic layout
that could perturb the respondent and harm the good run of the survey.
But you must not design dull and bad presented questionnaires, such as
the one in the left example. The right example, that has exactly the
same question, presents a sober structure but professional. Some online
surveys tools, such as the Net-Survey software that is used here,
proposed standards quality layouts by enabling you to easily adapt the
graphical interface of your company.
Let us here point to importance of keeping pages of the questionnaires
as light as possible. A page having big size images takes more time to
be loaded. This can make the questionnaire slower, especially during the
survey launching, if the surveys responsible person gets the bad idea to
request thousands of respondents at the same time. During the first
hours, many simultaneous connections bring a leverage of the heaviness
for the server, that is why it is important to minimize the load to be
treating by conceiving weight optimized pages for the questionnaire.
Rule n° 3: Well thought-out ideas are those that are explained clearly
In order to have correct and useful answers, you should ask clear
questions without any ambiguity. The internet user, who is alone behind
her screen, must perfectly understand what is asked to her and what
choices are proposed without having to do any particular efforts.
The questions (and the proposed answers) should exclusively use words
and expressions coming from the everyday language and understood by
everyone in the same way. The technical jargon, abbreviations and
complicated words must be avoided. If it is essential to use them, you
should give a clear explanation of their meaning (but short). Sentences
formulation also is important. The negative forms or
negative-interrogative are more difficult to be understood and should be
avoided. Also, conciseness is a needed quality, both for not losing the
respondent person and for giving a dynamic and short appearance to the
questionnaire. Questions should actually be expressed by short and
precise sentences. Ambiguous terms that could end to wrong answers
absolutely must be avoided. So, it is needed to replace terms such as
“Often” or “Rarely” by more precise scales such as “More than 3 times a
week” or “Less than once a month”, in order to avoid different
interpretation from one person to another.
In the same clarity idea, it is needed to deal with only one subject at
the time in one question. By asking the internet user judgment about the
“quality and prices of our products” you will be sure to get some biased
responses (is it about quality, price or quality/price relationship?).
Such a question must be divided in two or expressed in another way if it
is about evaluating the judgment of the quality/price relationship.
The logic in questions and responses proposals also is essential.
Unfortunately, it is common to have respondent people who find
themselves at an impasse to answer because of some questions that are
not adapted to their context and for which they actually do not find an
answer, or because of the incomplete list of responses that actually do
not cover their case. Temptation with such a situation is to simple
close the window (or to answer with any random response whether the
survey is paid).
Rule n° 4: Limit mandatory questions
This is not because you decided that the respondent absolutely must answer to all of your questions that you will have what you want. Conversely, the mandatory questions utilizations often make people bored, especially if the questionnaire is badly designed and that proposed responses do not cover all possible cases. Facing the impossibility to continue without answering the question whereas none of them is suitable, the only way out is the abandon (or answering with any random response, what more often happens than what we think). Rather than propose only mandatory questions, such we can often (too often) see, it is better to reserve mandatory question only to essential questions (the questions that cannot be avoided because the questionnaire would not be sensible). For example, this is the case with questions enabling the connection to different parts of the questionnaires. But even with this case, it is needed to provide a derivative response that will result in the end of the questionnaire (with a thank you message).
Rule n° 5: Optimize your tables
It is common to see many questions on tables in online questionnaires.
These questions follow on without break for the respondent person.
Usually, each table represents one question for the surveys responsible
person. And when it is mentioned at the beginning that the questionnaire
will propose around 10 questions, hundreds of responses are actually
expected from the respondent person.
Besides the importance to limit the global number of questions in the
survey, it is essential to avoid grouping tables with too many
successive questions. Actually, whether the structuration into tables
enables to make the evaluation easier, the multiplication of the lines
results in the weariness and the temptation to quickly answer, in a less
precise way.
We advise to limit on one hand the number of tables (no more than 3 or 4
in a survey) and to limit the number of items in a table (around ten in
each table). Whether the items consist in several lines sentences (what
you should avoid to do), their number must be reduced in order to obtain
a table that people can entirely see on a medium resolution screen
(today: 1024 x 768). This is actually important to have the row titles
visible for the respondent when she ticks the answer.
In order to make the finding easier on this table, it is essential to
alternate 2 different colors for pair and odd lines.
Some online surveys tools such as Net-Survey enable to make the table
easier by replacing the responses rows by rolling lists or, even better,
by imaging cursors. Beside the playful aspect, this last presentation
has the advantage to enable opposite presentations on tables with many
aligned items with a cursor for each of them (above and below) by giving
a feeling of simplicity and density.
Rule n° 6: Move from the general to the specific
The questionnaire organization is important to optimize the feedbacks.
Actually, a questionnaire beginning with complex or too much engaging
questions can discourage respondent people. In order to reassure
interviewed people and encourage them to answer, a “funnel-shaped”
questionnaire is needed, by starting with the easiest and less engaging
questions before moving to the most difficult and the most personal
ones. Unless you need to check quotas for the signing questions, these
questions would be better in the middle of the questionnaire than at the
beginning of the survey.
First questions need to be simples and clears. We can ideally begin with
questions giving a simple choice and having an obvious response for the
interviewed person, in order to make her feel confident. After this, we
can deal with factual questions (habits, behaviors) and then the opinion
questions (evaluation, satisfaction, recommendation).
About the satisfaction questions, it is advised to group the different
criteria to be evaluated into reasonable size tables (cf last slide). In
the idea of moving from the general to the specific, we need to place a
question about the global satisfaction before placing satisfaction
items. Besides the fact that this question is essential for some
advanced statistical analyses (ex: calculation of the importance matrix
compared to the satisfaction in Stat'Mania), its position before the
exposition of detailed elements to be evaluated enable to obtain a
global evaluation that is more exact and less subject to a modification
by the respondent person according to her responses to the different
items (whether they were placed before).
Rule n° 7: Guide the respondent people
In order to optimize the feedbacks, it is important to well welcome and
well guide the respondent person among the questionnaire.
First screen should present a greetings message and a brief but clear
explanation about what we want the respondent to do. It is also
important to precise the estimated time of the questionnaire by giving a
clear and sincere indication.
If you directly address to people that with who you already had a
contact (customers, prospects, collaborators…), you should tell about
the aim of the survey from the beginning and reassure about the
anonymous aspect of the results utilization. The surveys made on the
staff (360 degrees, social climate, stress at work, payment…) usually
are the most sensitive concerning these aspects. The use a faithful
third party (surveys institute or advising company) to design the
survey, or at least, to host it on external servers to the company,
enables to reassure collaborators and to get more reliable answers.
In the running of the questionnaire, it is advised to regularly give
indications about the way to fill questions (précising that the
respondent can tick several boxes for multiple questions for
instance).
Some filling mistakes should also be clearly indicated rather than
simply block the respondent people on the page by giving her the
temptation to abandon during the filling. If, for instance, mandatory
questions were not filled, it is needed to highlight them in a clear way
after having displayed a matching message (generally we circle them and
put a star or we change the color).
Finally, it is important to thank at the end of the questionnaire and to
give a link that enables to quit the questionnaire by automatically
closing the navigator window.
Rule n° 8: Place an advancement indicator
When a questionnaire exceeds 2 pages, it is important to give the
respondent person a precise idea about the screens number that she will
still have to display. To do this, the usual method consists in placing
on each screen an advancement bar that will indicate the percentage of
elements to be seen.
It is essential to avoid cheating on the given indication. However, in
some questionnaires that adapt to the respondent context, the
advancement is difficult to judge because pages to be passed depend on
the answers of the interviewed person. In this case, the indicated
advancement cannot exactly be proportional but can be optimized
according an average with lower or faster evolutions on some screens.
However, you absolutely must avoid an advancement bar that steps
backwards.
Rule n° 9: Test the questionnaire before publishing it
Generally, all serious surveys responsible people test the whole
questionnaire before diffusing it by imagining all different filling
cases and by verifying the successions, the conditional masking, the
rotations and other automatic fillings. However, we do not always think
about testing the questionnaire on different navigators. And so we
sometimes risk being surprised, even on small random displays but that
differently behave according to the navigator.
We advise to systematically verify your questionnaires on navigators
likely to be used by the target population (by also thinking about
testing them on different versions of each navigator that are still
used).
In France, the most used navigators (January 20132) are: Chrome
(32,15%), Firefox (28,72%), Internet Explorer (27,17%), Safari (9,72%)
and Opera (1,01%). You can consult actualized statistics and for the
geographical area that you want on the American Statcounter
website.
Rule n° 10: Correctly request your penpals
This step is essential to the success of the survey. At this level, you
must both take care of the message and of the timing.
About the message, an appeal to be a part of the survey should comprise
the invitation text itself with a direct link to the survey. Whether the
survey is anonymous or not, the link should be personalized in order to
directly connect the requested person to a questionnaire identified by a
code. This is the only efficient way to avoid a person to not answer
several times to the same survey by clicking on the same link. If a
later click happens, the system will be able to position the respondent
on the same questionnaire or to indicate her that she finished to answer
it. The alternative system that consists in using cookies is to be
prohibited because it only works for users who authorized cookies on
their device.
Message should of course be clear and have a well appearance. But this
is not necessarily useful to integrate images that risk to not being
able to be displayed on the screen. A simple text properly designed and
disposed often seems to be more serious and less “commercial”.
About the timing, it is useful to choose appropriate moment to address
the emails. A message sent to professional on a Friday night could not
be read on Monday morning because of the huge quantity of other emails
and tasks of the new beginning week. You should also avoid sending email
on Monday. For this population, Tuesday, Wednesday and Thursday should
be favored. For private individuals, Saturday is the day where opening
rate is the most important.
The spread of requests is highly recommended in order to avoid overloads
of servers that host the survey. Actually, the addressed messages are
usually opened within the minutes or hours following the sending, and
rarely after this time. If thousands of respondent people are requested
at the same time and get to the survey at the same time, slowing down
should be expected, whatever the powerful of the system. Some surveys
platforms such as Survey-Manager intend the spacing out of
the messages sending (a spacing out that is about some seconds
fractions) in order to avoid the overload as well as the risk of
becoming a spam…
