Five Actions to Achieve Higher Response Rates in Online Research

The importance of a sufficient response rate in surveys cannot be emphasized enough, since it is crucial for the representativeness of your sample and thus affects the generalizability, reliability and validity of the entire survey. Yet researchers are struggling to achieve appropriate response rates in online research, since the amount of various survey invitations and spam e-mails sent to people is constantly increasing. Fortunately, there are several actions a researcher can do to increase the probability of survey recipients to answer a survey.

Action #1: Design a Relevant and Inspiring Invitation Letter

• Provide a short description of the survey topic; the survey description should be long enough to arouse the respondent’s curiosity, whereas a too long invitation letter might prevent the recipient from reading it at all. The more relevant the topic is for the recipient, the more likely he or she is to take the survey.
• Tell the respondent how he or she benefits from taking part in the survey. What will be done based on the research results? Will the survey takers be rewarded with an incentive? Are you going to publish a summary of the survey results for the respondents?
• Inform about respondent anonymity, especially in cases where the research topic is sensitive.
• Don’t forget to mention the survey length in minutes. All respondents are afraid of monstrous surveys that might take an hour to respond to.

Action #2: Provide a Good Incentive

• An incentive is a good and commonly used method to increase the respondents’ motivation to take part in a survey. A generally used incentive is a price draw, where respondents can win products, discount coupons or gift cards. It is also recommendable to tell the respondents where and when the names of lottery winners are published and how the winners are contacted.
• Make sure that the incentive is equally useful for all respondents. To ensure this, you can offer the winners a possibility to choose from several prize alternatives.
• A good incentive can increase the representativeness of your final sample, since it might also motivate neutral respondents to take part in a survey.

Action #3: Design a Short and Simple Questionnaire

• Remember to include a progress bar in your questionnaire; showing how much is left of the survey will minimize the risk of survey takers dropping out of the survey when it starts to feel long.
• Avoid too many matrix questions in a row, since this might exhaust the respondents; instead try to apply a pleasant mix of different question types.
• Don’t design surveys that take more than 20 minutes to answer.
• Create a visually appealing survey, preferably including the company logo and a nice-looking layout.

Action #4: Choose the Publishing Date Carefully

• The response rates will not be optimal if you send out the survey close to a weekend or when people are having their annual leaves. It is also recommendable to avoid sending out invitations in the afternoon when the respondents might be preparing to go home after work.

Action #5: Leave Enough Time for Field Work and Send out Reminders

• Although the majority of respondents tend to respond within a few days, a good minimum length for the field work is one to two weeks. This will ensure that also respondents checking their e-mails more seldom have the possibility to answer.
• It is advisable to send out at least one reminder after 4 field work days. The respondents’ mailboxes might be crowded with other incoming letters, and someone might not have noticed the invitation letter or might not have had time to reply when the invitation was sent out the first time.

Annika Herrgård, M.Sc. Econ (Marketing)

The author has worked in market research for several years and is now working at Webropol as Research Solutions Manager.

The significance of n

Rarely one letter stirs up so much passion as ‘n’ in statistical research. Sample size is given considerable attention and a research with tens of thousands of responses is valued highly and considered to be absolutely reliable. A research with a few hundred respondents may be considered suspicious and the results may be (conveniently) ignored as nonsense or just a poorly selected sample. Whether the population size is a few hundred units or hundreds of thousands of units or whether statistically significant results are achieved does not seem to matter in these images.
When cross-tabulating and analyzing small subsets of collected data more data is obviously needed, but even then relevant results can be found on a much smaller n than usually imagined. For example, when comparing groups with more than 500 samples, statistical tests will essentially become obsolete. Virtually any difference is statistically significant. It shouldn’t be neglected, however, that this difference might not be relevant in any way. Interpreting these results might even be misleading. Reliable does not equal valid or relevant.

What is a relevant sample size?
Perhaps sample size should be approached with a slightly more open mind than just from the viewpoint of scientific accuracy. A researcher should always consider what is relevant for the research at hand. Of course it is understandable that, for example, in medical research or when monitoring manufacturing process of a large factory, margins of error should be extremely small. Even small errors can cause large damage to health or business. However, when researching already inaccurate phenomena, such as customer behavior, accuracy of the results just does not carry the same importance. When looking at people's intentions, opinions and other business issues, its more important to focus on relationship between different factors, observed trends and changes in these.

Sample size vs. Question formulation
What makes criticism of a small sample size particularly interesting, is that very rarely anything is spoken about question formulation or survey design. These, however, have much greater significance for accurate results than the amount of respondents. I will probably write more about this topic later on. Also the representativeness of the respondents and the population size is commonly ignored. Studies can be found where the amount of respondents even exceed the relevant target group size. When questions that are relevant for marketing decision makers, there is just no sense in targeting the survey to anyone somehow involved with the marketing function just to get more respondents.

Conclusion
My simple hint to a researcher who wants relevant results: focus more on what questions you ask, how you ask them and who is responding your survey than how many respondents you manage to lure. Accurate targeting also reduces people's perceived "burden of questionnaires" and in the long term will improve all of our chances to get answers to questions we feel are important.


Written by Petteri Pohto
Research Director, M.Sc. (Econ.)
Scan Survey Service

How can I prevent emails from getting caught by junkmail filters?

A quick guide for creating a good email template

When a new email message arrives, both email servers and email programs conduct several tests, on which they base their assumptions on whether the email is spam or not.

For example the widely used SpamAssassin method conducts almost 750 tests for each structure, content and sending information for each single message. For every failed test SpamAssassin assigns a certain amount of points, and when the sum reaches the alert level the message is classed as junkmail. Quite often even “real” emails end up in the junkmail folder specifically because of the structure and contents of the email.

In this article we will review the six most typical problems and the solutions to them.

Problem 1: Special characters in the email subjectThe highway to the junkmail folder is using numerous and repeated non letter characters, such as !?#€$%&(){[]}, in your email subject line.

Solution: Stick to text and only use necessary punctuation, such as commas and period marks. Avoid exclamation points and do not use currency symbols, such as £, € or $ - these you can easily replace with words (”pounds”, “euros”, "dollars”).

Problem 2: Low amount of text in relation to picture area
Junkmail filters are very critical of messages that contain very small amounts of text or when the amount of text is small compared to the area of the pictures you have used. Quite often this happens when trying to present the message of the email partly or fully in picture format.

Solution: Most of the contents should be typed in as text. Make a habit of using pictures only as graphic elements – do not put your main message in the pictures.

Problem 3: High numbers of line breaks in relation to the amount of text
In case your email contains a small amount of text but numerous line breaks, the spamfilter might suspect that you are trying to hide something towards the bottom of the email.

Eg. If your message contains only one line; ”Please answer this survey” and a link to a online questionnaire at the end, you should not put in lots and lots of line breaks in between.

Solution: In case your email does not contain lots of text, remember to keep an eye on the number of line breaks. Increase the amount of text slightly (this will most likely also raise your response rate!) Or simply delete unnecessary line breaks.

Problem 4: Links that seem to direct to other addresses than you see on the screen.

Your message is one step closer to junkmail if you have tried to make the message text look better by masking the URLs it contain as a link that is different from the actual destination.The problem occurs when the text visible to the respondent is e.g. www.acme.co.uk/survey, when in fact the URL leads to www.acme.com/AcmeuksurveyAugust2010

Solution: Never show a link as a diferent address than the actual URL is, instead use words, suchas as “Click here” or “Enter our website here”. Of course you can always choose to show the link in its original form.

Problem 5: Text with small font size & colours close to the background colour

This problem is common when unsubscribe link and respondent source information are tried to be made as unnoticeable as possible and then placed at the bottom of the email.

Solution: Do not use the smallest font size and avoid text colours that are close to the background color. Even with large font sizes junkmail fiters will regard your message as suspicious if the text colour is too close to the background colour, i.e. it thinks you are trying to hide something in the message.

Problem 6: Non-standard or badly written HTML code

In general, junkmailfilters are very doubtful towards messages that contain non-standard HTML code. The problem often occurs when writing an email template manually or using external editors. Even if a message created in an external editor looks good in the browser, it does not mean that the code is on all accounts good enough. This is because most browsers are optimized to show poorly written pages as well as possible.

Solution: If you create an email template manually, or using an external editor, it is always worth letting the Webropol ABC editor check the text. Copy the message in html mode to the Email message field, click on the ABC editor to open it and save. The Webropol editor will save the message structure in the best possible form.

A small case study: How to get followers on social media

As many marketers out there are still trying to figure out social media and how to grow their fanbase on Facebook, we thought we would share this story with you.

In July 2009 Mr Tapio Junes started a group, or a "cause", on Facebook against drunk driving. With very little effort his cause had virally spread to reach 42.000 followers in March 2010.

Starting the spread
In July 2009 Mr Junes started by googling the web for facts and statistics about drunk driving. He posted these on his Facebook “Cause page”. Tapio sent a message to his friends in Facebook. The message was simply a comment about these facts, “Are you annoyed by this too?” The spreading really took off when he started posting news items, video clips and statistics about drunk driving. He never actually sent direct messages to the followers, he emphasises that catchy news on group’s Wall is enough to get followers work for the common purpose.

Maintaining the growth
On a typical day 20 to 60 new followers join the cause, and statistics in Facebook show that many of these instantly attract 1 to 2 additional new members. All Mr Junes does anymore is post a new news items and related videos on the group wall every now and then but the viral spread is continuing. As this is a topic that engages many people it has also spurred active commenting on the pages.

So, let’s summarize what we learned from this case:
1. Start the spread with a short core argument to existing contacts. Make sure it has some news value to make it interesting and credible
2. If possible, support your messages visually. Choose images that have a similar impact as your core argument.
3. Be active and give members a reason to be active and keep coming back: The best reason to be is frequent and interesting updates, so look for interesting sources and always introduce your material by describing links. A great method for writing relevant descriptions is to simply ask yourself “why am I posting this link”.
4. Let the group members take care of inviting people, this occurs if you manage to keep the pages interesting and up to date.
5. Don’t “spam” people with messages, people do not like huge amounts of messages from the group owner
6. Celebrate growth of the group by thanking when you reach milestone numbers!
PS. (For your reference, the Finnish Anti Drink driving group can be found in Facebook on: http://apps.facebook.com/causes/108740 )

Webropol brings Text Mining to its online survey software

We have some exciting news for you! As first in the world, Webropol brings Text Mining capabilities to online survey software. Working as part of the Webropol toolset, the Text Mining solution allows for analysis, classification and grouping of textual answers based on occurring themes, keywords and respondent background data. Thanks to this toolset organisations can now process and analyse unstructured survey data more quickly, thoroughly, and objectively than ever before.

”It is unfortunate how often one runs into situations where managers would like to ask customers for free feedback, but will not do so in the fear of having to read sheet after sheet of answers. The Text Mining solution opens up completely new ways to use unstructured data for business intelligence purposes when hours worth of manual work is done in minutes.”, says UK Country Manager Mukesh Bassi from Webropol.

By comparing textual answers based on other responses and on background data managers get an insight into how feedback, suggestions, and ideas vary depending on e.g. employee satisfaction or customer life cycle. This will also help organisations set up and fine-tune their numeric customer and employee metrics to measure the things that are the most relevant.

”The Text Mining solution suits excellently both short surveys, and more complex research, especially when you are asking respondents for their ideas, feedback, or advice. It is virtually impossible to collect all ideas unless you gather answers in free form”, Mukesh points out.

The Text Mining solution works as part of the Webropol survey software, but data can be brought for analysis from external sources as well.


More information:
Mr. Mukesh Bassi
UK Country Manager, Webropol
telephone +44 (0) 24 7632 3229
email: firstname.lastname@webropol.com

Reach the world with viral surveys on Facebook

In our last blog post we discussed the possibilities of virally spread surveys in social media. After posting it we got some questions from people asking how they should go about to make the most of the fan pages they have on Facebook. For this we put together a video called ”Reach the world with viral surveys on Facebook”. Enjoy!


What’s the next step?
If you’re already using the Webropol survey software you can start sharing surveys on Facebook right away! Any survey created can be shared with fans and friends using the application. If you’re not using the Webropol survey software yet, you can start by making free and ad-free polls with the Facebook application.

Grab the application for free at http://apps.facebook.com/webropol

If you're interested in hearing more about the possibilities of virally spread Facebook surveys, don't hesitate to contact us! Also be sure to visit our Facebook fan page for more news and information!



Ps. If you for some reason cannot access the video above, it’s also available on Youtube.

Online viral surveys on Facebook

OR: How you turn 200 fans into a million responses at zero cost

We’re very excited to present a solution that will change online surveys forever. The new Facebook application from Webropol allows for viral spreading of surveys. This means you can harness the force of social media for product development, idea generation, feedback, and virtually any other kind of surveys. The application will help you turn 200 respondents into a million responses – at virtually no cost!

To be honest with you, even we think it’s quite a bold statement to say that you can somehow turn 200 people into a million responses. That’s why we start this blog post by putting our money where our mouth is.

Let’s say you have a modest Facebook fan base of 200 people and you send out a message with an online survey link attached. A response rate of 25% you will get you 50 responses. However a good result this is, it’s not using nearly the full potential of social media nor nearly reaching all 300 million Facebook users. What if you could somehow reach the friends of the respondents and the friends of those, and the friends of those...?


The force of viral spread
The average user of Facebook has 130 friends. This where the WEBROPOL application begins: when a person responds to a survey, she can show it on her news feed for her friends to see. When you factor in that 130 friends of each of the fifty who responded see the survey, you reach an additional 6500 pair of eyes. And if just a modest 5% of those take the sur vey, you have 375 responses more. Once again, the friends of those who responded see the survey in the news feed and answer. At just three steps from the original respondents, you could have tens of thousands responses.



















Should your viral response rate hit as high as 20 per cent, you would get a staggering million answers with just three levels!


How does it work?
Using the WEBROPOL application is a no-brainer. You simply create a normal survey and paste its link into the application. Then you go to your Fan Page and send an update to fans. There you can even target the recipients based on location, age and gender. Just click ”send” and wait for the responses to drop in!


Free to use and no ads!
The WEBROPOL application is 100% ad-free: there are no annoying ads, no pop-ups and no intrusive messages. It’s also worth mentioning that no information is passed on to any 3rd party. The application is absolutely free to use and there are no separate licence fees or hidden charges.

If you you’re not using the Webropol online survey software yet, you can still create simple polls with the Facebook application. Then just click ”Polls” and send out a poll to your friends!

Try out the Facebook application right away!



Glossary & Frequently Asked Questions

What’s a Facebook Fan Page?
In Facebook’s own words, a Fan Page is ” a public profile that enables you to share your business and products with Facebook users. When your fans interact with your Facebook Page, stories linking to your Page can go to their friends via News Feed. As these friends interact with your Page, News Feed keeps driving word-of-mouth to a wider circle of friends.” It’s completely free to set up and operate a fan page even for commercial purposes!

Just to get an idea of different fan pages, have a look at these: Coca-Cola, Pringles, Nokia, Webropol.

How many people are on Facebook?
According to Facebook its users have on average 130 friends. The number of Facebook users amount to over 300 million spread world-wide. Every day more than 8 billion minutes are spent in Facebook. For more information, see the Facebook statistics page.