Friday 22 October 2010

The significance of n

Rarely one letter stirs up so much passion as ‘n’ in statistical research. Sample size is given considerable attention and a research with tens of thousands of responses is valued highly and considered to be absolutely reliable. A research with a few hundred respondents may be considered suspicious and the results may be (conveniently) ignored as nonsense or just a poorly selected sample. Whether the population size is a few hundred units or hundreds of thousands of units or whether statistically significant results are achieved does not seem to matter in these images.
When cross-tabulating and analyzing small subsets of collected data more data is obviously needed, but even then relevant results can be found on a much smaller n than usually imagined. For example, when comparing groups with more than 500 samples, statistical tests will essentially become obsolete. Virtually any difference is statistically significant. It shouldn’t be neglected, however, that this difference might not be relevant in any way. Interpreting these results might even be misleading. Reliable does not equal valid or relevant.

What is a relevant sample size?
Perhaps sample size should be approached with a slightly more open mind than just from the viewpoint of scientific accuracy. A researcher should always consider what is relevant for the research at hand. Of course it is understandable that, for example, in medical research or when monitoring manufacturing process of a large factory, margins of error should be extremely small. Even small errors can cause large damage to health or business. However, when researching already inaccurate phenomena, such as customer behavior, accuracy of the results just does not carry the same importance. When looking at people's intentions, opinions and other business issues, its more important to focus on relationship between different factors, observed trends and changes in these.

Sample size vs. Question formulation
What makes criticism of a small sample size particularly interesting, is that very rarely anything is spoken about question formulation or survey design. These, however, have much greater significance for accurate results than the amount of respondents. I will probably write more about this topic later on. Also the representativeness of the respondents and the population size is commonly ignored. Studies can be found where the amount of respondents even exceed the relevant target group size. When questions that are relevant for marketing decision makers, there is just no sense in targeting the survey to anyone somehow involved with the marketing function just to get more respondents.

Conclusion
My simple hint to a researcher who wants relevant results: focus more on what questions you ask, how you ask them and who is responding your survey than how many respondents you manage to lure. Accurate targeting also reduces people's perceived "burden of questionnaires" and in the long term will improve all of our chances to get answers to questions we feel are important.


Written by Petteri Pohto
Research Director, M.Sc. (Econ.)
Scan Survey Service

Friday 9 July 2010

How can I prevent emails from getting caught by junkmail filters?

A quick guide for creating a good email template

When a new email message arrives, both email servers and email programs conduct several tests, on which they base their assumptions on whether the email is spam or not.

For example the widely used SpamAssassin method conducts almost 750 tests for each structure, content and sending information for each single message. For every failed test SpamAssassin assigns a certain amount of points, and when the sum reaches the alert level the message is classed as junkmail. Quite often even “real” emails end up in the junkmail folder specifically because of the structure and contents of the email.

In this article we will review the six most typical problems and the solutions to them.

Problem 1: Special characters in the email subjectThe highway to the junkmail folder is using numerous and repeated non letter characters, such as !?#€$%&(){[]}, in your email subject line.

Solution: Stick to text and only use necessary punctuation, such as commas and period marks. Avoid exclamation points and do not use currency symbols, such as £, € or $ - these you can easily replace with words (”pounds”, “euros”, "dollars”).

Problem 2: Low amount of text in relation to picture area
Junkmail filters are very critical of messages that contain very small amounts of text or when the amount of text is small compared to the area of the pictures you have used. Quite often this happens when trying to present the message of the email partly or fully in picture format.

Solution: Most of the contents should be typed in as text. Make a habit of using pictures only as graphic elements – do not put your main message in the pictures.

Problem 3: High numbers of line breaks in relation to the amount of text
In case your email contains a small amount of text but numerous line breaks, the spamfilter might suspect that you are trying to hide something towards the bottom of the email.

Eg. If your message contains only one line; ”Please answer this survey” and a link to a online questionnaire at the end, you should not put in lots and lots of line breaks in between.

Solution: In case your email does not contain lots of text, remember to keep an eye on the number of line breaks. Increase the amount of text slightly (this will most likely also raise your response rate!) Or simply delete unnecessary line breaks.

Problem 4: Links that seem to direct to other addresses than you see on the screen.

Your message is one step closer to junkmail if you have tried to make the message text look better by masking the URLs it contain as a link that is different from the actual destination.The problem occurs when the text visible to the respondent is e.g. www.acme.co.uk/survey, when in fact the URL leads to www.acme.com/AcmeuksurveyAugust2010

Solution: Never show a link as a diferent address than the actual URL is, instead use words, suchas as “Click here” or “Enter our website here”. Of course you can always choose to show the link in its original form.

Problem 5: Text with small font size & colours close to the background colour

This problem is common when unsubscribe link and respondent source information are tried to be made as unnoticeable as possible and then placed at the bottom of the email.

Solution: Do not use the smallest font size and avoid text colours that are close to the background color. Even with large font sizes junkmail fiters will regard your message as suspicious if the text colour is too close to the background colour, i.e. it thinks you are trying to hide something in the message.

Problem 6: Non-standard or badly written HTML code

In general, junkmailfilters are very doubtful towards messages that contain non-standard HTML code. The problem often occurs when writing an email template manually or using external editors. Even if a message created in an external editor looks good in the browser, it does not mean that the code is on all accounts good enough. This is because most browsers are optimized to show poorly written pages as well as possible.

Solution: If you create an email template manually, or using an external editor, it is always worth letting the Webropol ABC editor check the text. Copy the message in html mode to the Email message field, click on the ABC editor to open it and save. The Webropol editor will save the message structure in the best possible form.

Friday 26 March 2010

A small case study: How to get followers on social media

As many marketers out there are still trying to figure out social media and how to grow their fanbase on Facebook, we thought we would share this story with you.

In July 2009 Mr Tapio Junes started a group, or a "cause", on Facebook against drunk driving. With very little effort his cause had virally spread to reach 42.000 followers in March 2010.

Starting the spread
In July 2009 Mr Junes started by googling the web for facts and statistics about drunk driving. He posted these on his Facebook “Cause page”. Tapio sent a message to his friends in Facebook. The message was simply a comment about these facts, “Are you annoyed by this too?” The spreading really took off when he started posting news items, video clips and statistics about drunk driving. He never actually sent direct messages to the followers, he emphasises that catchy news on group’s Wall is enough to get followers work for the common purpose.

Maintaining the growth
On a typical day 20 to 60 new followers join the cause, and statistics in Facebook show that many of these instantly attract 1 to 2 additional new members. All Mr Junes does anymore is post a new news items and related videos on the group wall every now and then but the viral spread is continuing. As this is a topic that engages many people it has also spurred active commenting on the pages.

So, let’s summarize what we learned from this case:
1. Start the spread with a short core argument to existing contacts. Make sure it has some news value to make it interesting and credible
2. If possible, support your messages visually. Choose images that have a similar impact as your core argument.
3. Be active and give members a reason to be active and keep coming back: The best reason to be is frequent and interesting updates, so look for interesting sources and always introduce your material by describing links. A great method for writing relevant descriptions is to simply ask yourself “why am I posting this link”.
4. Let the group members take care of inviting people, this occurs if you manage to keep the pages interesting and up to date.
5. Don’t “spam” people with messages, people do not like huge amounts of messages from the group owner
6. Celebrate growth of the group by thanking when you reach milestone numbers!
PS. (For your reference, the Finnish Anti Drink driving group can be found in Facebook on: http://apps.facebook.com/causes/108740 )