Absolutely 100% agreed! I’m so not in favor of dictating, or framing things as right and wrong, either. You’ll get to disclose, articulate, educate, etc. and your customers will get to take all that into consideration as they benefit from the great service you offer. Sounds good to me 🙂

Hi Christine – Glad you found the response useful and we will certainly try to include as much documentation including perspective about sample size, great reinforcement. We do have to be a little careful though about dictating how many people is the “right” number to ask or what confidence level is “right” for the decision you are making. What I mean is that not everyone is only doing statistically relevant studies, sometimes as marketers, PR people, Product Managers, etc (not just boring market researchers like me) we just want to get a feel for what people might say or take a quick pulse as a directional insight that lines up with other types of market research. We also have to be realistic that if what I am deciding on his a $1000 campaign I can’t spend more than $50 on measurement etc versus a much larger campaign. Great thoughts all the same, looking forward to keeping the conversation open as well.

Amy, thank you for pinging Adam ~

Adam, thank you *very* much for taking the time to pen such a thoughtful reply. I played with the calculators on SurveySystem, and get the sense that a sample size of 100 wouldn’t generate confident results for any population above 150 or so.

You might already have this idea in your roadmap, or are doing it already: displaying the confidence interval for users after they’ve designed their survey and before the hit “go”. I’m thinking of Priceline saying, “Hm, for a hotel with this star level in this neighborhood, you have a 20% chance of getting the room for that price.”) The difference here is that the quality of the room is the same whether the customer is paying $50 or $500. For Qriously, in many ways, the quality of what is being purchased might be significantly affected by the price.

I see an important obligation to be upfront about that, and am hopeful that you’ll be able to blend full disclosure of WHAT people can measure with Qriously and HOW meaningful those measurements can be. If anything, that disclosure should drive up demand for larger sample sizes and smaller populations, i.e. higher priced – and some would say higher quality – surveys.

Thanks again for the informative comment here and openness to keep the conversation going ~
Christine

(Adam from Qriously) Hi Christine – great question and thanks Amy for the ping. We haven’t published as much documentation about how to apply the data as we probably should and would love there to be a few external champions to fill in some of the gaps (any takers?). Most of our existing b2b clients are market researchers or consumer insight professionals already so this is clearly something we’ll have to get better at as we come out of closed beta.

I won’t bore everyone to death with measurement science but wanted to offer a few points about polling and lighter forms of market research. I’ll start by saying that there are three big elements to any market research approach: speed, depth of insight and price. Something like Qriously falls in the very fast and low cost spectrum offering directional (lighter) insights. A focus group which has much more granularity takes longer and comes with a much larger price tag. I am sure there are lots of opinions out there about the balance of which approach is right but I’ll offer a few examples of where this type of quick approach might make sense.

For instance, a question that Qriously might answer well for a National organization is around issue sensitivity such as (in honor of our World Wildlife Fund contributor):

How important is wildlife preservation to you?
Answers options: “Not really” AND “Very!”

Or when an organization wants to pre-test a campaign concept:

“WWF: Protecting the Future of Nature”
Answer options: “No way!” AND “Keep the slogan!”

Then you might look at the results where concentrated areas of “for” vs “against” opinions are and better plan your online, TV, radio or in person media efforts. One of the core benefits of a speedy research approach vs a more traditional one is that you can run it throughout a campaign and watch the effect of what you are messaging (then make changes on the fly).

You also asked a great question about sample size or how many people to get responses from. Finding the right sample size to ask or being able to base decisions on a particular response is more subjective than you would think. What I mean is that the number you should ask really needs to tie into how much internal weight you are putting on the number and needs to be a balance for cost. Many times polling style research is used as a compliment or pre-cursor to other more granular types of research like a focus group.

Instead of me pasting everything I thought this site did a neat job of helping you come to the right conclusions: http://www.surveysystem.com/sscalc.htm

Let me know if you have more thoughts or would like to chat via skype.

Thanks!
Adam

Christine – That’s a great question! You can see some of the contextual
guides in the screen shots of the results. But I’m sure that’s a question
Adam can answer – I’ll ping him!

Thanks!