Why Some Crowdsourcing Challenges Fail

Crowdsourcing campaigns present a variety of unique opportunities for engineers, researchers, and weekend hobbyists. As such, one vital ingredient to success is having a well-crafted, insightful challenge design.

Unintended barriers to entry, unclear or unengaging details, inappropriate time frames, and lack of meaningful incentives are among the chief offenders to an otherwise well-intentioned challenge.

Fortunately, proactively recognizing how and when to address these troublesome issues is the best approach to ensuring your crowdsourcing challenge reaches a successful completion!

Unclear, Uninteresting Problem Statements

The goals of a crowdsourcing or crowd engineering challenge are commonly referred to as problem statements; poorly phrased problem statements––typically bogged down by unnecessary jargon or ambiguous verbiage––make it difficult for your target crowd to decipher the challenge.

An effective problem statement outlines the desired outcome of a challenge while simultaneously convincing others that your crowdsourcing project is lucrative and/or worthwhile.

One study extolls the benefits of effectively communicating challenge value; it increases visibility, encourages crowd participation, and potentially garners the support of sponsors, corporate entities, or government agencies. In addition to showcasing challenge goals in a compelling manner, problem statements need to be pragmatic. Unrealistic goals often lead to failure, so it is paramount to set reasonable objectives.

To solve a complex problem, multiple challenges honing in on various aspects of the problem may prove more fruitful.  

The Devil’s in the Details

Challenge details get into the “nitty-gritty” of a project and require meticulous crowdsourcing design.

These key details may be public or only available to pre-screened experts, such as software engineers or academic researchers. Especially when initiating crowd engineering challenges, providing enough detail to understand the problem and parameters is vital.

It’s also equally important not to overcorrect in the other direction. Overly specific details can limit the diversity of submissions, so a careful balance must be struck between too much detail and too little. This middle-ground approach will facilitate solutions that address unique challenge goals while also giving contributors room for creativity.

Challenges can also fail when their details do not excite the community.

Put simply, contributors don’t want to devote their precious time and resources to a boring (static) project. One popular campaign strategy is to generate initial buzz by presenting your own ideas first, and challenging potential contributors to discuss or even “one-up” your idea. Getting the ball rolling yourself lessens the “classroom” anxiety over being the first to submit a concept, and may help get some momentum going early in the game.

Details that do not address audience motivations can also lead to challenge failure. Focus in on your target crowd and consider whether cash prizes, prestige, moral incentives, or some other award will motivate them the most.

Unreasonable Challenge Durations

While setting reasonable time frames for crowd engineering may seem obvious, challenges often fail to take many factors that affect timing into account.

Size of the target community, complexity of the challenge, contributor incomes, or skills required to solve a problem can all influence the duration of a challenge. For most projects, contributors will not be pursuing challenge prizes as a primary source of income, so they will likely only devote 5 to 20 hours per week.

Larger communities also generate ideas more rapidly than smaller ones. When diverse skills, such as mechanical engineering and graphic design, are needed, there must be ample time for contributors to form teams.

Human psychology also comes into play with deadlines. Challenge participants often procrastinate if the time frame is too long, submitting last-minute, haphazard solutions. However, if the challenge duration is too short, contributors will not have enough time to submit.

In general, a challenge period of four to eight weeks has shown optimal results.

Lack of Prizes and Incentives

Prizes are, not surprisingly, a major factor in predicting challenge success.

Consider what will attract the most contributors, and no, it isn’t always money. Prize breakdowns, the distribution of rewards amongst successful contributors, are also vital to crowdsourcing design. Prize breakdowns need to give contributors a decent chance at winning, especially when complex solutions are required. If only the top three out of a massive crowd can win, contributors may not even put in the work.

Skilled participants quickly drop off if the odds of winning are too low to justify their time and effort.

When offering financial rewards, carefully consider the amounts. Meager rewards may not motivate the crowd, but high cash prizes can actually intimidate contributors, making them doubt their chances of success. In crowd engineering, monetary incentives for expert solutions must be on par with in-house compensation. This may seem counterintuitive to those looking to use crowdsourcing as a “cheap” option, but remember that incentives are not awarded until contributors have successfully completed the challenge.

A one-time award for desired results pales in comparison to the costs of continual in-house research and development. Remember: crowd engineering is not cheaper—it is more cost effective.

Barriers to Entry

Outreach methods are a frequent barrier to entry; with poor outreach, a project simply cannot find its target crowd.

Expert contributors also often face barriers when a challenge seeking their skills tries to crowdsource with a gig-economy approach: offering low incentives, short time frames, and unclear details.

However, the success of NASA challenges has shown the immense potential for crowd engineering, although the organization’s prize pool budget is a major factor in its success. To avoid alienating talented contributors, challenges must have clarity, sufficient detail, ample time, and incentives that properly compensate experts for their efforts.  

Finding Success in Failure

Finally, it is also important to remember that value can be found in failure.

Even if a challenge remains unsolved, it is foolish to throw out all ideas generated, which could prove useful in other applications. Additionally, in crowd engineering, discovering what doesn’t work ultimately brings engineers closer and closer to finding viable solutions for future projects.

Valuable networking connections are often created through the trials and tribulations of failed challenges, as well as opportunities to incite new avenues of research and development that otherwise may never have come to light.

Have YOU experienced any of these problematic issues in recent crowdsourcing challenges? Are there any we missed? How did you handle these hurdles, and did you succeed in overcoming them?

We’d love to hear your experiences! Share with us below!