Shameless plug for the service we built to solve this for ourselves - https://www.pickfu.com
The MVP was to meant to replace asking randos at a coffee shop about your idea/design/etc. Over time we've added demographic targeting, follow-up questions, and other requested features. Nowadays online sellers and other creators use it like an online focus group to validate ideas and products.
Some other ideas for finding an audience for your MVP:
- google the exact problem you're trying to solve, reach out to authors/bloggers who've written about it
- online ads to your target demographic
- LinkedIn prospecting (if you have a target customer profile)
- Subreddits of your target problem area
- Start writing/tweeting about the problem space and engage the audience that follows
- comments section of relevant YouTube videos
Actually, there is a way you can view the votes by (gender,age-group,etc.) - click on the "Option A vs. Option B" tab right under "Who responded to your question?" Is that what you were looking for? If so, it's our fault for not making it more visible.
Thanks for the heads-up about the mturk job search - we're making tweaks so that those previews show a sample question from now on. Helps keeps the responses more random too.
No, "Option A vs Option B" isn't quite what I was looking for. Let us stipulate that, once I see the results, I decide that I don't care what guys think of my photo. I just want to compare the difference in response between 18-34-year-old women, 35-49-year-old-women, and 50+-year-old-women. I don't see a way to get that information from your current interface.
Ah yeah, I get what you're talking about. It's one of the features I've been wanting to add for a while too - filtering the responses by any demographic combination. Not pri 1, maybe in the next version. =)
We looked at using Qualifications a bit, but quickly decided that it wasn't worth our time to deal with them, at least for PickFu in its current state.
I quite like that it's ultra-simple right now. 1 price, few options. Not something you need to think about too much. Just do it.
I'm not saying that choosing your own demographics, number of participants, more complex surveys might not be useful sometimes. But I love that I can get my head around this in 10 seconds.
Our respondents don't get paid unless they actually explain their answers - which both reduces false responses and provides unique value and insight to the service.
I still think doesn't help. I liked the idea and did the turk for one of your questions (this was my first time doing the turk, btw). I was asked to visit a myspace webpage, listen entirely to 2 songs and say which one I like better and why. All this for 3 cents? I did it, cause I am a good fellow but I have hard time thinking of people who are willing to spend 10 minutes of their time for 3 cents.
the FAQ example is a bad example: I can always answer "yes" and then supply as reason "every questions I had was answered". No need to read the FAQ page, of course.
I think if you want to insert any kind of validation system you would have to count the time it took for the turk to answer your question and make sure we are in that range for every hit.
As noted before, it's tough to know that 100% of the responses are truthful on any survey. With PickFu, you can read the explanations to get a feel as to which respondents put some thought in.
I'm sorry you had a rough experience with that specific music-listening job - we've never had a more time-consuming question submitted. Most are in the order of "which site do you prefer?", like this current one: http://pickfu.com/AGSIBT. These get answered far faster by the MTurk workers.
another one of the creators here, thanks for the warm comments.
a follow-up question - how much more compelling would this be if you could choose/influence the demographics and # of responses (potentially at a higher cost)?
I would personally find it more compelling, but I'd suggest that it shouldn't be a premium feature.
When it comes to consume research, demographics are everything. Statistical significance of sub-samples can plummet quickly.
If I'm marketing a product specifically to, say, 30-44yo men, it would be valuable to restrict the sample exclusively to those men and I think i'd have a better chance of seeing statistical meaning in the sub-samples I'm looking for, eg, income, marital status, etc.
It seems to me now that I'd have to post it "at large" and the 30-44yo demo I'm looking for would itself be a sub-sample.
I am not a marketing professional, but as a small business owner its one of my many hats. I have used both Maritz and, more recently, Pinecone, and I'm probably wrong but I don't remember even being presented with an option to open the surveys to everybody.
And what I'm saying is.. I wouldn't even consider using your product without that ability. You have a fantastic 1.0, though, so I'm not knocking the product, just offering my $0.02.
And some demographics of your own: We've got a dozen of us on the payroll with 2009 sales looking to be a bit over $2MM.
And no joke, if I were you, I'd eat your dogfood and put up a survey to determine the importance of this feature!
EDIT:
I do want to add that I get that your site is supposed to be intro-level market research. But if you're going to limit to 50 replies, demographic targeting would be very important to me, for anything other than a "should we get thai or pizza for lunch" questions.
If you do let me collect 500-1000 replies, I'd probably have more statistically meaningful sub-samples, so as long as you give me a nice drill-down interface I'd find it useful, even if i couldn't target my question.
To take the other side...there is currently a beautiful simplicity about this idea and execution.
I pretty much just fall asleep when I start trying to deal with A/B testing, market research, ad effectiveness, etc. This is so simple that all I have to ask myself is, "Do I have a question I'd like to ask a bunch of people?" and "Do I have five dollars?" That's a really good place to be, because everybody in business has questions, and everybody has five dollars.
I will absolutely use this site. I do question where the people are coming from, which I guess is a demographic question...but I think any time you ask of the user to choose it should be later in the sales process. For statistics wonks, mention it in the FAQ, or something. But I wouldn't clutter up the initial offer with too much stuff like that. I wish I had a product that was this simple of a sell.
And, of course, the more involved things get, the more expensive it will have to be. I really like five dollar market research. I might not be so enthusiastic about $25 market research.
Fair enough... but if this is just a completely random, not-even-remotely-scientific man on the street approach, then what problem does this solve that a poll i can add to my website/blog/favorite forum via Fantastico in 180 seconds doesn't?
It puts it in front of people that don't already know about your product.
I can ask usability questions of my users, but their existing knowledge will effect what is "easy" and "intuitive" (intuitive generally means, "what I'm used to"). I can ask marketing questions of them, but I've already pulled them into my site, so my current message will be the one that gets reinforced. I can ask "compare these two products" questions, but I already know a preponderance of people on my site prefer my product to our competitors (one of which is Fantastico, by the way). I can ask any number of things, and we do frequently ask our customers opinions on things, but by virtue of them being on our site the results are known to be skewed in exactly the way we don't want them to be for a large number of questions.
Also, I don't want to clutter up my site with constant polls. When someone is on our site, we have very clear goals we want to achieve: Teach them about the benefits, help them choose the right products, show them how to use it, answer their questions. Asking them random questions isn't my idea of good design, when it comes to those goals.
Example: I am designing a logo for a market research site. I put up the pickfu logo against mine (or just use two of mine), & ask users which company does market research.
Examples 2-100: A poll on my site or blog would work fine but... I don't want people on my site to know what I'm polling. I don't have a site. I don't know how to do polls on my site. My IT department says it will take 8 weeks. My web design guys say they don't have the module for this particular CMS - they can build one from scratch for $2k. I told the web design guys to go to hell & found an embeddable polling service, it'll take 4 weeks & they won't be responsible if it doesn't work. I worked it out with the web design guys but my boss says we need to use the companies standard colour scheme - the widget won't change its colour.
I really like the site, and it is well executed. I think I'll certainly use it at some point. Good price point.
I completely agree that there needs to be some way to restrict who answers questions.
Another example would be if someone was making a Twitter application, they might want to restrict people who answer the question to only those that use twitter.
Or if you were doing something aimed at webmasters, you might want to restrict the responses to people employed as such.
As also mentioned in other comments, use your poll system to do your own poll on how desirable this could be.
I would be a little wary about reading into comments about this. Immediate thoughts brought on by a quick browse are often different from while-using wishlist items. There are also a wide range of possible uses for this. I am guessing mine is different from other people's.
I thought of using this as part of a sales process for consulting services. That would put you in some sort of competition (or symbiosis) with compete.com. For that, immediately obvious conclusions (however simple), low price & easy to use (time is an issue here) are all essential. precision, advanced options & such, not so much.
It would require more effort to get 50 responses from a tiny demographic segment than a large one, so I was wondering if having the price inversely correlated to the segment size was reasonable to an end user.
The MVP was to meant to replace asking randos at a coffee shop about your idea/design/etc. Over time we've added demographic targeting, follow-up questions, and other requested features. Nowadays online sellers and other creators use it like an online focus group to validate ideas and products.
Some other ideas for finding an audience for your MVP: