Survey sites like Swagbucks have often a prescreening mode in which one is asked questions like your annual income, whether you own car or not. It is observed that most of the time if one selects options denoting lower income, zero car, then the respondent disqualified and no point/incentive given to the respondent (to be precise, Swagbucks gives 1 SB but that is far less than if one qualifies and completes the survey). This should lead many respondents to opt for higher income, thereby distorting the survey results. So why these surveys do not take into consideration this fact and do away with prescreening?
$\begingroup$
$\endgroup$
3
-
3$\begingroup$ There may be some logic behind it, but we do not know the motives of a specific company, so I recommend rephrasing the question slightly. $\endgroup$– Frans RodenburgCommented Jul 9 at 11:48
-
2$\begingroup$ Does Swagbucks disclose the eligibility factors in order for a survey participant to continue? I would hope not. This is the kind of "deceit" - hopefully approved by an internal review board - that helps ensure collection of valid data. If the participants don't know that income is an eligibility factor for receiving the higher compensation, any subsequent reason to lie about income is hopefully not related to the survey design. $\endgroup$– AdamOCommented Jul 9 at 12:40
-
4$\begingroup$ @AdamO Apparently there are entire communities dedicated to this survey-for-cash stuff. It seems to be a bit of a running joke that they offer higher-rewarding surveys to oddly specific (disclosed) groups, often related to income: 1, 2, 3 $\endgroup$– Frans RodenburgCommented Jul 9 at 15:34
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
Presumably the people commissioning surveys are mostly interested in answers from only certain demographics... probably people with disposable income, because these are the people who are even able to afford a product the survey is about.
And they quite understandably don't want to pay for answers they are not interested in.
Therefore, it makes some sense to prescreen.
And yes, this may incentivize people to lie. Then again, if you terminate early (and get less money), you spend less time on this particular survey and can continue on to the next task.
answered Jul 9 at 11:51
$\begingroup$
$\endgroup$
Results from surveys conducted on a specific target audience can likely be sold for much larger amounts then surveys conducted on anyone willing to reply.
Furthermore, being able to analyze how different demographics reply differently is likely also where part of the value of these surveys comes from.
Hence, there is a trade-off between specificity in the participants and compromising the validity of the replies.
(Highly specific surveys vs inadvertently incentivizing heavily to fake user information.)
answered Jul 9 at 12:05
$\begingroup$
$\endgroup$
There's possibly a principal-agent problem going on. Swagbucks' clients want to survey respondents with demographics X. Swagbucks wants to be able to tell the client that they have many individuals self-reporting as that demographic. These interests are not in perfect alignment.
Telling survey respondents in advance what demographics are needed to advance in the survey will increase the number of individuals self-reporting as having those demographics (advancing Swagbucks' short term pecuniary interests), at the cost of encouraging misleading responses. The client understands that they must rely on self-reported demographic information, and understands that such data has intrinsic limitations. However, they may not understand (fully) the substantial increase in limitations that results from the current scheme, and in my view it would be easy to frame these concerns as just part of the limitations of online surveys when dealing with clients of low to medium statistical sophistication.
To be clear, this is not based on factual information I have about Swagbucks or any related entities. I have no nonpublic information about their business practices, and frankly I haven't read much of the public info. I'm simply pointing out an abstract sociological fact arising from a possible imbalance of statistical expertise and using Swagbucks as a hypothetical instantiation. I am making no factual assertions about their business practices, of which I am mostly ignorant.
PS, as you might imagine, Swagbucks does not present themselves to their clients by that name; Swagbucks is the name of a platform, which is provided to its products (i.e. the survey respondents). Here is what I believe to be one of the pages meant for the clients who would be the ones submitting surveys, where they refer to themselves as Prodege. We see that they advertise being able to provide access to (attractive, pool-side) individuals with specific demographic criteria.
Whether or not this is advertising access to respondents of their Swagbucks platform or some other platform is unclear to me at first glance.
