Managing Crowds in
Innovation Challenges

Arvind Malhotra
Ann Majchrzak
Crowdsourcing for innovation is typically conducted as an “innovation challenge.” Despite the popularity of innovation challenges, there appears to be a growing consensus that innovation challenges do not succeed at generating solutions with competitive advantage potential. This article presents three ways in which managers can assure that their innovation challenges are fruitful: foster different crowd roles to encourage contribution diversity; offer knowledge integration instructions and dual incentives; and offer explicit instructions for sharing different types of knowledge.
(Keywords: Creative Collaboration, Innovation, Creativity, Crowdsourcing, Open Innovation)

I

nnovation challenges, also known as innovation tournaments and idea contests, are a manifestation of crowdsourcing.1 When running an innovation challenge, a company posts an open call on the Web to solicit solutions from a diverse range of individuals. For example, GE’s Innovation Challenge solicited new technologies for its sustainability product line and a Lego Challenge asked the public to suggest unique Lego products as new revenue streams. By 2017, over half of consumer goods producers are projected to employ crowdsourcing for 75% of their consumer innovations.2
Despite the popularity of innovation challenges, there is a growing consensus that the current manner in which innovation challenges are implemented fails to yield solutions that provide the competitive advantage potential sought by the companies sponsoring the Challenges.3 Competitive advantage potential as judged by executives at the company sponsoring the Challenge means that the solutions proposed by the crowd provide new strategies and alternatives that can help the

The authors would like to thank NSF for their generous funding of this research (NSF #121983),
Solomon Darwin and Sirkka Jarvenpaa for their help with finding sponsors for the challenges, and
Claudia Kubowicz Malhotra and the anonymous reviewers for their helpful comments through the review process.

CALIFORNIA MANAGEMENT REVIEW

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

103

Managing Crowds in Innovation Challenges

Arvind Malhotra is the T.W. Lewis
Distinguished Scholar and Professor of Strategy and Entrepreneurship at the
Kenan-Flagler Business School,
University of North Carolina at Chapel
Hill.
Ann Majchrzak is Professor of Digital
Innovation at the Marshall School of
Business, University of Southern
California, and at ESADE Business School,
Universitat Ramon Llull, Spain.

company differentiate itself from competitors. The failure to deliver solutions with potential for generating competitive advantage potential has led some analysts to argue against the use of innovation challenges when new perspectives on ill-defined problems (such as new business models or new sources of revenue) are needed. As a result, innovation challenges are increasingly relegated to a form of buzz marketing and customer engagement.4 The research presented in this article indicates that these conclusions about the limitations of innovation challenges may be premature. Crowds are able to develop solutions with competitive advantage potential, but only when the crowd is asked to go beyond just the sharing of their ideas. Achieving the full potential of innovation challenges requires the challenge participants to also integrate their knowledge with the knowledge of others in the crowd. Understanding Knowledge Integration in Crowds
The research question central to this article is: How can crowds be encouraged to integrate their knowledge to create new solutions with competitive advantage potential for a company?
Knowledge integration can be used to solve well-defined as well as ill-defined problems.5 Ill-defined problems, also referred to as wicked problems,6 are those that can be interpreted, understood, and structured in a number of different ways, leading to multiple alternative solutions. Asking the crowd to develop new products, services, and business models are examples of ill-defined problems. For ill-defined problems, solutions are more likely to surface when the knowledge is combined across different perspectives, such as different disciplines, experiences, and responsibilities.7 Therefore, solutions with competitive advantage potential are more likely to surface when the crowd is asked not only to share knowledge, but also to integrate the knowledge they share. For example, the combination of a corporate strategist’s knowledge of the market and a manufacturing engineer’s knowledge of new technologies is more likely to yield a strategically insightful solution than only the ideas generated by each individually. Most innovation challenges today ask the crowd to post their individual ideas. Rarely are participants asked, encouraged, or enabled to combine their ideas into integrated solutions.
The literature on knowledge integration8 identifies a number of different actions needed as part of a knowledge integration process. Applying this literature to a knowledge integration process specifically for innovation challenges suggests that three actions are needed: sharing, highlighting, and combining. These actions are illustrated in Figure 1.
The first action in the knowledge integration process—knowledge sharing— requires participants in innovation challenges to share the knowledge they have about the innovation challenge question. In many innovation challenges, participants are only asked to “post an idea.” This focus on posting an idea leads to individuals

104

UNIVERSITY OF CALIFORNIA, BERKELEY

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

Managing Crowds in Innovation Challenges

FIGURE 1. Knowledge Integration Process in Innovation Challenges

KNOWLEDGE INTEGRATION PROCESS
SHARING
• Posting ideas, examples, facts, tradeoffs • Commenting

HIGHLIGHTING
• Voting on others’ posts
• Promoting others’ comments

COMBINING
• Creating solutions by combining knowledge in multiple posts

not posting other types of knowledge that they may possess about the innovation challenge question. Participants in innovation challenges may know facts, have relevant personal experiences, know of examples of other companies facing a similar challenge, or point out the key tradeoffs that need to be addressed in order to resolve the innovation challenge question. Because innovation challenges do not generally explicitly ask participants to share this knowledge, it is usually not shared. Therefore, participants may need to be explicitly encouraged to share the diverse knowledge they possess. Sharing diverse knowledge creates “a collage of diverse perspectives on the problem,”9 enabling participants in the innovation challenge to see the breadth of concerns and issues needing to be resolved for generating solutions with competitive advantage potential.
The second action in the knowledge integration process for innovation challenges is highlighting the shared knowledge most relevant to developing solutions to address the innovation challenge. Not all knowledge shared needs to be incorporated into solutions. For example, a post about a minor change to an existing product line may not need to be integrated into a solution about a new strategic direction for the company. Knowledge highlighting is critical to innovation challenges because the amount of shared knowledge organically offered by the crowd can rapidly become overwhelming and disorganized. Highly relevant posts can be buried in this disorganized collage; thus preventing the most relevant knowledge from gaining attention by the crowd for inclusion in solutions.10 In most innovation challenges, participants voting “up” a post or “promoting” a comment can highlight relevant knowledge. Innovation challenge websites typically rank order the posts based on the number of votes received so that those with the most votes are more likely to be seen by more participants. Therefore, the guidance provided to the participants about the criteria to be used for highlighting a post becomes critical in determining if the innovation challenge will produce integrative solutions. For example, in many innovation challenges, participants are asked to vote on a post based on whether they “like” it. However, “liking” is an ambiguously subjective criterion and may not be an indicator of how relevant the post is to a potential solution. When knowledge integration is desired, highlighting needs to be based on the potential of the post to contribute to a solution. For example, a challenge participant might personally “like” the post as a product to purchase, but may not believe that it would change the competitive situation of the company.

CALIFORNIA MANAGEMENT REVIEW

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

105

Managing Crowds in Innovation Challenges

Highlighting based on potential contribution to a solution should then help ensure that more participants see the posted knowledge, making it more available for integration.
The third action in the knowledge integration process for innovation challenges is knowledge combination. In many innovation challenges, participants are asked to merely post their ideas and leave it up to the participants to offer comments to refine others’ ideas. As a result, participants rarely combine either previous ideas or previous knowledge that has been shared and highlighted by the crowd. A knowledge integration approach to innovation challenges differs by explicitly encouraging participants to create solutions that combine either ideas or other relevant knowledge that has been previously shared by other participants.11 The research presented here suggests that managers interested in increasing the quality of their crowd-created solutions should design their innovation challenges as a three-action knowledge integration process involving knowledge sharing, knowledge highlighting, and knowledge combination.

Description of the Research
The first phase of the research involved interviews with innovation managers, representatives of innovation intermediary companies, and innovation software vendors. From these interviews, the current state of the practice for managing innovation challenges was distilled. In this article, the current state of practices is referred to as ‘conventional’ innovation challenges. In the second phase of the research, two sets of small-scale experimental innovation challenges were designed. These innovation challenges were conducted on behalf of sponsor companies as pilots for experimenting with new ways to manage crowds during innovation challenges. Sponsor companies participated with the objective of obtaining solutions from the crowd that had greater competitive advantage potential than the conventional innovation challenges they had conducted previously.
The experimental innovation challenges introduced mechanisms for encouraging knowledge integration actions. Instructions were developed and innovation challenge software was reconfigured to guide participants through the knowledge integration process. Instead of instructions that would be typically conveyed in conventional innovation challenges, participants were explicitly asked to first share diverse knowledge they possessed about the innovation challenge question. The participants were also asked not simply to vote on posts based on “liking,” but to vote explicitly based on their views about how useful the knowledge would be for formulating a solution to address the innovation challenge question. Participants were also explicitly asked to combine either ideas or relevant knowledge that was shared by other participants so as to develop “integrative solutions.” These sets of instructions are different from typical instructions for conventional innovation challenges that only encourage posting of ideas, voting on ideas based on “like,” and do not provide clear instructions or encouragement for integration.
The two sets of experiments in this research differed in their specificity. The first set consisted of four matched-pair innovation challenges. Four companies simultaneously sponsored two innovation challenges (with the same innovation challenge question): one with conventional instructions and the other with the knowledge integration instructions. In this first set of experiments, knowledge

106

UNIVERSITY OF CALIFORNIA, BERKELEY

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

Managing Crowds in Innovation Challenges

integration instructions explained how to follow the knowledge integration process, but did not specify any particular types of knowledge that participants should share. In the second set of experiments, consisting of two additional innovation challenges, participants were provided instructions about how to integrate knowledge, as well as additional instructions about the different types of knowledge that should be shared, highlighted, and combined.
The experimental innovation challenges in this research not only differed from conventional innovation challenges in terms of instructions, but also in the incentive structure. In conventional innovation challenges, monetary prizes are the primary incentives offered. Such incentives focus exclusively on rewarding the “best” ideas as voted by the crowd and expert judges.12 In contrast, with knowledge integration, participants need to be willing to engage in actions not previously expected in innovation challenges: allow their knowledge to be used by others, use others’ knowledge in a transparent way, highlight others’ shared knowledge for relevance in integration, and perform the cognitively difficult task of integrating others’ knowledge. Therefore, incentives were introduced that rewarded participants not only for sharing their own knowledge, but also for developing, highlighting, and combining the knowledge contributed by others. The Top Contributor Board of conventional innovation challenges, which is based on the number of ideas contributed, was replaced with a Top Collaborator Board. Names of top collaborators were displayed based on actions they took to help develop, highlight, and combine others’ knowledge. Top collaborators received equivalent monetary awards ($75-$300) as those participants contributing to the winning integrative solutions, as determined by a panel of judges from the company. In addition, innovation challenge participants were offered non-monetary awards, e.g., the opportunity to meet with the expert judges to present their integrative solutions, certificates of recognition, and sponsor company apparel. Consistent with research on incentives to foster collaboration in online communities,13 the incentive system used in the experimental innovation challenges was a dual-incentive structure with both small monetary awards and recognition awards (see Sidebar 1 for details of the experiment).

Sidebar 1: Details of Experimental Context of the Research
First Set of Field Experiments
The first set of field experiments was conducted to test whether the introduction of our knowledge integration guidelines, and incentives to use those guidelines, affected the competitive advantage potential of solutions generated by the innovation challenge participants. Four companies were involved. For each company, two challenge websites were created: one for the knowledge integration (treatment) approach and the other for the conventional approach (control group). The conventional approach encouraged posting ideas only, and providing awards to initiators continued on next page

CALIFORNIA MANAGEMENT REVIEW

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

107

Managing Crowds in Innovation Challenges

of the best ideas with no awards for collaborators. Therefore, we had four matched pairs of innovation challenges.

How the Innovation Challenge Participants Were Attracted
Two of the four companies invited only employees to participate; the other two invited external parties. For the two employee challenges, innovation managers from the companies helped identify representative groups from across the multiple functions and helped market the challenge to a diverse set of functional groups within the company. For the external challenges, to obtain participants, multiple venues to find individuals interested in the specific topic of the challenge question were identified. For example, for the software challenge, multiple listservs of software programmers were identified. To ensure the same people did not participate in both the control and treatment innovation challenges, each venue was randomly assigned via a particular link in the announcement to either the treatment (explicit knowledge integration instructions) or conventional challenge (no explicit knowledge integration instructions). Participants were blind to which condition they were in. Analysis of participants’ log data after the challenge was completed indicated minimal posting across the experimental and control challenges. Students in an innovation class were used as confederates to provide initial idea seed activity in equal measure across the control and the treatment challenges at the start of the challenges, and were intended to reassure participants that the challenge was an active one.

How the Posts in the Innovation Challenges Were Coded
Each “top level post” and “thread comment” was systematically coded for whether it included an integrative solution, defined as “solutions that combine either ideas or other relevant knowledge that has been previously shared by other participants.” Each post was also coded for whether it contained specific knowledge types: facts, examples, or tradeoffs.
The two authors first coded 30 posts individually. Then the codes were compared with each other. Wherever there was disparity between their codes, the authors then discussed the difference in their interpretations. Once a common agreement was achieved, the authors proceeded to code the rest of the posts. An inter-rater agreement of 94% was achieved, with disagreements between the coders discussed and the differences in interpretation was resolved.

Second Set of Field Experiments
In the second set of experiments, two companies agreed to conduct innovation challenges. This allowed us to test the value of differentiated knowledge categories on innovation. In both challenges, participants received the same knowledge integration process guidelines as in the earlier set of experiments. In addition, they also received guidelines for knowledge contribution categories. The guidelines and categories are shown in the following table. continued on next page

108

UNIVERSITY OF CALIFORNIA, BERKELEY

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

Managing Crowds in Innovation Challenges

Guidelines

Incentives

Leaderboard

Experimental
Aim

1. Please post in the following Categories:
Facts, Tradeoffs,
Examples, Wild Idea
Seeds, Integrative
Solutions
The categories were listed on the navigation menu appearing on each page

Outcome-based
(Top 3 ideas) as well as Processbased (Top 3
Collaborators).
Both received similar monetary and nonmonetary rewards. Top
Collaborators

More integrative solutions produced by the challenge participants. Greater participation (more top level posts, more comments, more votes on top level posts as well as comments)

2. When you post, indicate which category your idea belongs in.
3. Vote on posts and comments based on how useful they are to formulate a solution to the challenge. Two different challenge questions were posed by the two companies.
In the first challenge, company-wide employees were invited to participate in the challenge. The employees were from across multiple functions in the company. The innovation managers from the companies helped identify representative groups from across the multiple functions and helped market the challenge to a diverse set of functional groups within the company. For the second challenge, students attending a class on open innovation identified online discussion groups and online forums whose populations would be interested in the challenge. Repeated postings were made in these forums and discussion groups to attract participants for the challenge.

The experimental innovation challenges were conducted in partnership with innovation managers at six different companies, with each company providing an innovation challenge question. Examples of innovation challenge questions were:
• What are some of the services-led strategies that our company can adopt to transition from a hardware- and software-centric business to a servicescentric business that creates new markets and new customers?
• What new and innovative services/products can our company provide to its customers using new mobile technologies?
• What technology could the company offer its retail and brand clients that would enable consumers to enjoy all the benefits of online shopping
(i.e., reading ratings and reviews, social product recommendations, leaving a review themselves, etc.) while walking through physical stores?

CALIFORNIA MANAGEMENT REVIEW

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

109

Managing Crowds in Innovation Challenges

Results of the Innovation Challenge Experiments
There were 5 findings from our analysis.

Finding #1: When participants in innovation challenges were given explicit knowledge integration instructions, they generated higher-rated solutions than participants in innovation challenges who were given conventional instructions.
The objective of the experiments was to test the impact of explicit knowledge integration instructions on the ability of innovation challenge participants to develop higher-rated solutions. Table 1 shows the explicit instructions for knowledge integration compared to the instructions for conventional innovation challenges. To assess the effect of knowledge integration instructions, log data available from the innovation challenge websites regarding participants’ knowledge integration actions were compared across two different sets of innovation challenges. In one set of innovation challenges, the participant received explicit knowledge integration action instructions; and, in the other set, participants received conventional instructions. The number of comments made by participants was used to measure the knowledge integration action of knowledge sharing. The number of votes on comments was used to measure the knowledge integration action of highlighting. The outcome of the knowledge integration action of combining was measured based on the quality of solutions developed. Three company stakeholders from each of the four companies rated the quality of solutions developed. The company stakeholders were not involved in the challenge but were familiar with the context of the innovation challenge.
The judges were given integrative solutions that were generated in their innovation challenges. The judges were blind to whether the solutions had been developed by innovation challenge participants given conventional instructions or knowledge integration instructions. They were asked to rate (on a 7-point

TABLE 1. Knowledge Integration vs. Conventional Instructions
Conventional Instructions

Knowledge Integration Instructions

1. Start by posting an idea

1. Post knowledge to stimulate others

2. Tweak/modify ideas posted by others

2. Add your knowledge (as comments) to modify others’ knowledge

3. Provide feedback by commenting on others’ ideas

3. Integrate others’ knowledge to form complete
4. Follow the etiquettes of posting comments solutions (critique don’t criticize); encourage others to post
5. Vote for each idea indicating if you like or do not like 4. Encourage others to integrate knowledge the idea

110

UNIVERSITY OF CALIFORNIA, BERKELEY

5. Vote on posts and comments based on how useful they are to formulate a solution to the challenge VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

Managing Crowds in Innovation Challenges

scale) the extent to which each solution, if implemented, was likely to lead to a new strategy and offerings that would help the company differentiate itself from competition. As shown in Sidebar 2, the participants in innovation challenges with explicit knowledge integration instructions exhibited more comments per participant, and more votes per participant, compared to participants in challenges with conventional instructions. In addition, as also shown in Sidebar 2, solutions generated by innovation challenge participants with explicit knowledge integration instructions were rated higher than solutions generated by participants in innovation challenges with conventional instructions.

Sidebar 2: Differences in Challenge Participant
Behavior due to Explicit Knowledge
Integration Instructions in Conjunction with Dual Incentives
Impact on

Group

Mean (S.D.) F (d.o.f.)

Competitive
Advantage
Potential of
Solutions

3.11 (0.83)
Innovation Challenges with
Conventional Guidelines
Crowd (nideas=12)
4.14 (1.04)
Innovation Challenges with
Knowledge Integration
Guidelines Crowd (nideas =10)

6.65* (1,20)

* p < 0.05

Impact on

Group

Mean (S.D.)

Conventional Guidelines
Innovation Challenge
(n=140)
Knowledge Integration
Guidelines Innovation
Challenge (n=282)
Conventional Guidelines
Innovation Challenge
Comment
Promotes (Voting (n=140)
Knowledge Integration on Comments
Guidelines Innovation per Participant)
Challenge (n=282)

Number of
Comments (per
Participant)

F (d.o.f.)

0.34 (0.88)
5.88* (1,420)
1.12 (3.75)

0.72 (1.33)
6.21* (1,420)
2.18 (6.47)

* p < 0.05

CALIFORNIA MANAGEMENT REVIEW

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

111

Managing Crowds in Innovation Challenges

Finding #2: Crowds in the challenge often create solutions through natural discussions; consequently, solutions may be buried in knowledge exchange threads.
To understand how integrative solutions evolved over time, all posts in the innovation challenges were systematically coded for type of knowledge shared and whether the post was a solution or not (see Sidebar 1 for details of the coding).14 Posts in innovation challenges are organized as threaded discussion forums. Top-level posts indicate the start of a discussion and comments on top-level posts are the discussion threads. Participant voting can highlight top-level posts or individual comments. Each highlighting through voting brings attention to and attracts more participants to the threads. Commenting and voting on others’ posts should highlight knowledge embedded in threads for potential integrative solutions, which should then lead participants to post integrative solutions as new top-level posts.
Surprisingly, regardless of whether the crowd received conventional or knowledge integration instructions, participants did not always post solutions as a new top-level post. Rather, solutions were often buried as comments within a thread started by someone else. In fact, across the four matched-pair innovation challenges, of the 22 out of 107 posts that were coded as solutions, 20 of them were not located in top-level posts and were instead buried as comments in a thread.
Table 2 contains an illustrative example of a threaded discussion with solutions within the thread.

TABLE 2. Example of Ideas and Integrative Solutions Embedded in a Thread
Top-Level Post #1

Comment #1

Comment #2

Comment #3 (Embedded Solution)

Comment #4 (Embedded Solution)

112

Retail Banking reminds you of brick-and-mortar and fixed hours and ATM machines even though bulk of us do online banking and billpay. ATMs were an innovative “service” and automated banking. Can banking go the route of Borders bookstores?
What services and technology can our company and its partners bring to virtualize banking?
Major banks already have comprehensive online banking services, so it wouldn’t be like the case of Borders; most of the banks that have the physical presence are the same ones that provide the online services. Unless you meant something else by virtualization?
Thanks John. I was thinking we could eliminate the physical locations altogether. Why do I need to visit ATMs periodically to get cash (yes, we’ve eliminated hassles of bill-pay via online services. we can deposit checks from phones these days). Still the solutions seem to address the banking/payment spectrum partially. How can technology revolutionize banking?
I agree that online banking would be much easier. A problem with anything that uses wireless connection and sensitive materials: it can be hacked and/or stolen. [Company] could lead the online banking world if they created a way to increase security. They could also invest in making a security app for Google, Android, and Apple because there are not many security systems for mobile devices as complex as the security on a computer.
Thanks Sam. So we fix security, we create an app and the right security at the device level. What about cash or virtualizing it
(e-cash instead of real currency?).

UNIVERSITY OF CALIFORNIA, BERKELEY

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

Managing Crowds in Innovation Challenges

Buried solutions in discussion threads suggests the need for a new form of highlighting that helps identify when a solution has been offered within a thread.
Unburying solutions enables other participants to turn their attention to refining and possibly extending those solutions in new ways. Therefore, this finding suggests that crowds of challenge participants do not fit into convenient categories that companies have conventionally used to categorize contributions from crowdsourcing. The crowd can contribute by developing, highlighting, and combining others’ ideas and knowledge rather than merely contributing their own ideas as top-level posts.

Finding #3: Innovation challenge participants clustered into distinct profiles consistent with different types of knowledge integration actions they performed.
Participants rarely spend much time posting or reading all the posts in an innovation challenge. Therefore, it is unlikely that they would engage in all three of the actions of the knowledge integration process. Yet, in the small group literature from which much of the empirical research on the knowledge integration process has been developed,15 the assumption is that all team members will exchange their knowledge, help to highlight knowledge important for integration, and participate in combining the shared knowledge into new solutions. The notion that all actions may not be performed by the same set of participants has not been explored previously. It might be the case that as long as each action in the knowledge integration process is performed by at least some of the participants, the crowd as a whole may be able to combine that knowledge into new solutions.
Therefore, participation in the four matched-pairs of innovation challenges was analyzed to determine if the three knowledge integration actions were performed by different groups of individuals.
A total of 294 participants were registered across the four matched-pair innovation challenges. Those participants who engaged in at least one of the knowledge integration actions were included in this analysis. This led to conducting the analysis on 73 active participants to determine if there were distinct knowledge integration action profiles.16
The following types of posts were examined for each participant to identify distinct knowledge integration action profiles:
Knowledge Exchange Actions
• facts, examples, and/or tradeoffs as comments in others’ post threads
• facts, examples, and/or tradeoffs as comments in their own post threads
• questions on others’ threads encouraging more discussion
• comments on others’ top-level posts made at the same time as either posting a new top-level post or replying to others’ comments on own top-level post
Knowledge Combining Actions
• solutions as comments in others’ post threads
• solutions as own top-level threads

CALIFORNIA MANAGEMENT REVIEW

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

113

Managing Crowds in Innovation Challenges

Since all active participants voted, highlighting was not a differentiating knowledge action in these profiles. Through a cluster analysis, four distinct clusters (or “sub-crowds”) of participants were identified (see Sidebar 3 for details).

Sidebar 3: Cluster Analysis for Innovation Challenge
Participant Behavior
A two-step cluster analysis was conducted to explore how active participants in the innovation challenge grouped together based on their informational behavior—extending discussions that result in novel ideas by posting of facts, tradeoffs, and examples. Each post in the challenge was coded for whether it contained facts, tradeoffs, and examples about the challenge. Posts were also coded if they contained integrative solutions—solutions that combine either ideas or other relevant knowledge that has been previously shared by other participants. A two-step cluster was used for analysis because it can handle categorical variables (as opposed to other cluster analysis methodologies that require continuous variables). Further, using the model-choice criterion, the methodology automatically suggests the ideal number of clusters. The six knowledge integration action behaviors mentioned in the main text were used to drive the clusters (as shown below). Based on the
Akaike Information Criterion (AIC), a four cluster solution with cluster quality > 0.8 was suggested. The value of each of the cluster variates for each of the clusters is shown below.

Innovation Challenge Participant Clusters
Drive-byPosters

114

Knowledge
Benefactors

Cheerleaders

Post their idea suggestion as a top level post only, do not comment on others’ knowledge threads

Post solutions on others’ threads
Post on others’ threads at the same time they post their own idea as a top level post.
Post their own idea as a top level post
Comment on others’ thread and provide facts, examples, and tradeoffs. Comment on own thread with facts, examples, and tradeoffs
Comment on others’ threads with questions

Convenience
Generatives

Do not start their own knowledge threads, post ideas, facts, tradeoffs and examples in others’ knowledge threads

Do not post their own ideas. Comment on other threads’ but in a nongenerative way (no facts, examples, tradeoffs, or solutions) (n=22)
NO
(100%)
NO
(91%)

Post their own ideas as top-level posts. At the same time they comment on others’ knowledge threads and provide facts, examples, tradeoffs, and solutions
(n=8)
YES
(87.5%)
YES
(100%)

(n=30)
YES
(93.3%)
NO
(100%)

(n=13)
NO
(100%)
NO
(100%)

YES
(100%)
NO
(100%)

YES
(63%)
YES
(100%)

NO
(73%)
YES
(60%)

NO
(100%)
NO
(100%)

NO
(100%)

NO
(100%)

NO
(100%)

NO
(100%)

NO
(100%)

NO
(100%)

NO
(100%)

NO
(100%)

UNIVERSITY OF CALIFORNIA, BERKELEY

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

Managing Crowds in Innovation Challenges

The first sub-crowd, consisting of 30% of the active participants, is labeled
“Drive-by-Posters.” These are individuals who share their knowledge only once as a top-level idea post. They do not continue the discussion by replying to others’ comments on their thread or comment on others’ ideas or comments. In the knowledge integration process, their primary focus appears to be in conveying their knowledge as an idea post. They clearly are not knowledge integrators, per se. Nevertheless, by offering even one of their own ideas, they provide other participants with a new perspective to consider for integration that would otherwise not have been available.
The second sub-crowd, consisting of 11% of active participants, is labeled
“Convenience Generatives.” These are individuals who are much more actively engaged in the knowledge integration process than the Drive-by-Posters. They share their knowledge, highlight others’ knowledge, and combine previously posted knowledge into solutions. They comment on others’ knowledge contributions in ways that are positive and constructive, i.e., “generative.” However, the timing of their contributions indicates that their actions are not based on an unlimited altruism. They only post on others’ threads when they have logged in to the innovation challenge to check on the status of votes and comments made to their own posts. Thus, while Convenience Generatives are engaged in all three actions of the knowledge integration process, they do so only when it is convenient to them.
The third sub-crowd, accounting for close to half of the active participants, is labeled “Knowledge Benefactors.” A distinguishing feature of this sub-crowd is that the participants share their knowledge exclusively by commenting on others’ posts. They never offer top-level posts of their own solutions, but instead offer solutions as comments within someone else’s thread. Unlike Convenience Generatives, they repeatedly return to the innovation challenge to offer comments even if there have been no replies to comments they have provided previously.
Unlike Drive-by-Posters, they return often. They are actively engaged in all the actions of the knowledge integration process.
The final sub-crowd is labeled “Cheerleaders,” consisting of 18% of all active participants. These individuals do not share their own knowledge, limiting their knowledge integration actions to offering supportive comments on others’ contributions. For example, these individuals might say “great idea!” or “I really like where you’re going with this idea.” With their comments, these participants serve the important role of highlighting knowledge for others to attend to.
In essence, each of the four sub-crowds differed in how they engaged in the knowledge integration process. Knowledge Benefactors and Convenience
Generatives engaged in a greater variety of knowledge integration actions, whereas Drive-by-Posters and Cheerleaders each engaged in only one of the knowledge integration actions. Taken together, though, the four sub-crowds counter-balance each other and collectively perform all of the knowledge integration actions. When Drive-by-Posters initiate knowledge sharing, Knowledge
Benefactors and Convenience Generatives sustain the knowledge sharing and engage in combining the shared knowledge. Cheerleaders highlight knowledge,

CALIFORNIA MANAGEMENT REVIEW

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

115

Managing Crowds in Innovation Challenges

an action without which shared knowledge would be hard to combine. The Driveby-Posters and Convenience Generatives share their own knowledge, albeit at their own convenience. Knowledge Benefactors share knowledge exclusively by commenting on others’ contributions. Cheerleaders help highlight knowledge shared by others that they believe is particularly relevant to the innovation challenge question. Finally, Convenience Generatives and Knowledge Benefactors periodically return to the innovation challenge to combine participants’ knowledge into solutions.

Finding #4: Knowledge integration in innovation challenges may depend on which sub-crowds are engaged in the innovation challenge process leading up to the solution.
To further understand how the participants generated solutions varying in judges’ ratings, the proportions of sub-crowds for the innovation challenges with conventional and knowledge integration process instructions were compared. There were significantly more Knowledge Benefactors (increase of 70%) and Convenience
Generatives (increase of 52%) in the challenges with knowledge integration instructions than in the challenges with conventional instructions. Additionally, the subcrowds did not differ in propensity to post more highly rated solutions. This suggests that integration may be dependent on which sub-crowds are engaged in the innovation challenge process leading up to the solution.
For each solution posted, participants posting immediately preceding the solution were examined to determine if the majority of those posting were
Knowledge Benefactors or Convenience Generatives (versus Drive-by-Posters or Cheerleaders). A significant difference was found. When the majority of posts preceding a solution were made by Knowledge Benefactors or Convenience Generatives, solutions were judged higher compared to solutions for which the immediately preceding posts were made by Drive-by-Posters or Cheerleaders. These findings suggest that
Knowledge Benefactors and Convenience Generators help crowds to combine knowledge, even if they do not post the combined knowledge as new top-level solutions themselves.

Finding #5: Providing specific instructions about which types of knowledge should be shared is positively associated with a higher level of knowledge integration process actions.
In the second experiment, two additional small-scale innovation challenges were conducted. In this experiment, participants were instructed to share four specific types of knowledge: facts about the challenge, tradeoffs, examples, and seeds. In addition to specific instructions on the home page of the challenge related to types of knowledge to share, a label option was implemented in the electronic submission form that required participants to choose which knowledge type their posts fit into before submission.
The innovation challenge participants were also encouraged to highlight solutions that had been specifically constructed from previously posted ideas by providing them with a separate label type, called “integrative solutions.” By

116

UNIVERSITY OF CALIFORNIA, BERKELEY

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

Managing Crowds in Innovation Challenges

specifically differentiating distinctive knowledge types, participants could more easily identify the contributions they felt comfortable to share. In addition, the shared knowledge was more organized and accessible to other participants.
Finally, integration was facilitated by explicitly asking participants to combine knowledge in previous posts.
As was the case with the knowledge integration process instructions in the first experiment, participants were asked to vote on posts and promote comments based on how useful they thought the posts were in formulating a solution to the challenge rather than in terms of how much they “liked” the post. Table 3 displays the instructions for the second experiment.
To assess the impact of highlighting specific knowledge types, the actions that participants took during the two innovation challenges of the second experiment (in which instructions were based on both explicit knowledge types and knowledge integration actions) were compared against all participants in the first experiment of the four matched pair innovation challenges (in which the instructions did not specifically mention the need to contribute according to specific knowledge types).
The participants were compared in terms of their knowledge sharing actions, measured as the number of top-level posts per participant and number of comments per participant. The knowledge highlighting actions were measured as the number of votes on top-level posts per participant and number of votes on comments per participant. The results (see Sidebar 4) indicate that, compared to innovation challenges with no specific instructions about types of knowledge to share, participant activity levels were higher in the two innovation challenges with specific instructions about types of knowledge to share. On the average, the number of top-level posts per participant was 2.5 time higher, the number of comments per participant to top-level posts were 2 times higher, the number of votes per participant on comments and top-level posts was 4 times higher and each participant was twice as likely to develop solutions (vis-à-vis participants in innovation challenges with no explicit knowledge type instructions).

TABLE 3. Instructions for Types of Knowledge to be Shared
Please post in any of the following categories:
• Facts—Information you have related to the challenge question.
• Tradeoffs—Issues or conflicts that could make a solution hard or impossible to achieve simultaneously
(e.g., low cost and hard to manufacture).
• Examples—Illustrations or demonstrations that link the challenge to your own experiences, or connect across business contexts that may otherwise seem unrelated.
• Seeds—Short statements that present early ideas that may serve as a catalyst for a comprehensive solution to the challenge.
• Integrative Solutions—Solutions that integrate across previously posted ideas and/or challenge relevant knowledge to solve the challenge question.
When you post, indicate which category your post belongs in.
Vote for posts based on how useful they are to formulate an integrative solution to the challenge.

CALIFORNIA MANAGEMENT REVIEW

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

117

Managing Crowds in Innovation Challenges

Sidebar 4: The Effects of Specific Instructions for
Types of Knowledge to be Shared,
Contribution on Knowledge Integration
Process, and Outcomes in Innovation
Challenges
Impact on

Group

Mean (S.D.) F

Number of
Integrative
Solutions (per
Participant)

0.10 (0.31)
Challenge Participants with explicit knowledge categories for contribution
(n=353)
Challenge Participants with no explicit 0.05 (0.23) knowledge categories (n=204)

4.27*

* p < 0.05

Impact on

Group

Mean (S.D.) F

Challenge Participants with
Number of Top- explicit knowledge categories
Level Posts (per for contribution (n=185)
Challenge Participants with no
Participant)
explicit knowledge categories
(n=237)
Challenge Participants with explicit knowledge categories
Number of for contribution (n=185)
Comments
Challenge Participants with no
(per Participant) explicit knowledge categories
(n=237)
Challenge Participants with
Number of Votes explicit knowledge categories for contribution (n=185) on Top-Level
Challenge Participants with no
Posts (per explicit knowledge categories
Participant)
(n=237)
Challenge Participants with
Number of Votes explicit knowledge categories for contribution (n=185) on Comments
Challenge Participants with no
(per Participant) explicit knowledge categories
(n=237)

0.72 (2.18)
7.96**
0.30 (0.68)

1.20 (4.33)
3.95*
0.59 (1.65)

2.96 (7.76)
18.43***
0.71 (1.97)

2.96 (8.28)
17.01***
0.70 (1.39)

* p < 0.05, ** p < 0.01, *** p < 0.001

Management Implications
Experiments are seldom conducted with innovation challenges because sponsoring companies are rarely willing to systematically manipulate how they manage the crowd. In this article, the research with companies willing to experiment indicates that crowd management matters. When crowds are managed, innovation

118

UNIVERSITY OF CALIFORNIA, BERKELEY

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

Managing Crowds in Innovation Challenges

challenge with little practical value can turn into ones that generate solutions with high potential for competitive advantage for the sponsor company. The results described earlier offer three managerial practice suggestions for managing crowds in an innovation challenge to achieve this benefit.

Managerial Practice Suggestion #1: Foster Different Crowd Roles to Encourage Contribution Diversity
Key Points
• Managers should worry about ensuring that there is enough diversity in the crowd. A new view on obtaining diversity should be considered—called “contribution diversity.” This view is based not solely on ensuring different forms of expertise, as is the case with conventional innovation challenges, but also ensuring different knowledge integration actions are undertaken as well.
• Four different sub-crowds, exhibiting unique types of knowledge integration process actions, should be encouraged to participate:
• Drive-By-Posters, who initiate discussions with the first post of discussion threads, forming the basis for further knowledge sharing by the crowd;
• Convenience Generatives, who help develop others’ ideas when it is convenient to them, playing a part in integrating the crowd’s knowledge to generate solutions;
• Knowledge Benefactors, who help develop and highlight others’ knowledge, and combine others’ knowledge to generate solutions; and
• Cheerleaders, who contribute supportive comments, thereby highlighting others’ knowledge relevant to the challenge
• Managers of innovation challenges should design incentives to encourage all four types.
As individual crowd members are unlikely to spend considerable time contributing to the innovation challenge, managers should make it possible for the crowd to engage in knowledge integration in a manner that can be customized to each crowd member’s passion and time. The crowd can, and should be, managed to allow individuals to perform their own preferred roles in an emergent knowledge integration process. Having different sub-crowds perform different knowledge integration actions within the crowd is important for generating better solutions. Therefore, managers should carefully monitor participation in innovation challenges to note absence of critical sub-crowds. Further, managers must take action to fill gaps by inviting more participants to join the innovation challenge and incentivizing them to engage in any underperformed knowledge integration actions, such as commenting or voting.
Having differential incentive structures for the sub-crowds will be critical for encouraging all four sub-crowds to join.17 Cheerleaders, who share little of their own knowledge but encourage others, may be motivated by the excitement of being part of something new. Knowledge Benefactors, in contrast, may be motivated by altruism, since they are offering their knowledge to help others develop their ideas without expecting knowledge in return. Drive-by-Posters clearly are motivated by the rewards because they do not contribute to develop or integrate others’ knowledge.
Thus, the greater they feel they have a chance of winning, the greater the likelihood

CALIFORNIA MANAGEMENT REVIEW

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

119

Managing Crowds in Innovation Challenges

that Drive-By-Posters will participate. Convenience Generatives only help others when they are already contributing to their own ideas. Therefore, they may need explicit incentives for continuing to perform the “inconvenient” task of helping others.
Managers should recognize that the incentive structure for innovation challenges needs to be designed to encourage different types of knowledge integration actions.

Management Practice Suggestion #2: Offer Knowledge Integration
Instructions and Dual Incentives
Key Points
• Innovation challenge instructions should describe expectations for the crowd of how to engage in knowledge integration.
• The crowd should be offered dual incentives (outcome-based and knowledgeintegration-process-based) to engage in knowledge integration rather than focusing merely on contributing ideas as top-level posts, because better solutions may emerge at any point in the innovation challenge as the crowd comments on top-level posts and builds on each other’s knowledge. Also by encouraging voting on comments, not just on top-level posts, knowledge will be surfaced that is useful for formulating a solution to the innovation challenge.
A dual incentive structure should be designed for innovation challenges. In addition to rewarding outcomes based on best solutions, managers should also reward contributions to the knowledge integration process: posting knowledge, commenting on and integrating others’ knowledge, and voting on posts and comments to highlight knowledge. In conjunction with knowledge integration instructions, dual-incentives can encourage more posting by Knowledge Benefactors and
Convenience Generatives, who perform critical roles within the crowd and are largely responsible for knowledge integration.
When only outcome rewards for best solutions are offered, there is less collaboration among the crowd.18 Giving rewards for the best solutions tends to drive crowd members to only share their personally derived ideas, rather than knowledge that could be integrated by others. Incentives should be designed with the objective of the knowledge integration process. Such incentives should encourage the crowd to offer comments in a way that provides new knowledge, makes tradeoffs more salient, and spurs solutions through combination of knowledge in previous posts.
Further, most technology platforms today are configured to allow crowds to vote only on top-level posts. The research in this article suggests that unless technology platforms for innovation challenges are configured for highlighting, crowds will miss and not utilize the knowledge and solutions that emerge in the course of the innovation challenge. As a consequence, the crowd will fail to highlight this knowledge for later combination. Managers need to reward and technology-enable those who highlight knowledge for integration.

Managerial Practice Suggestion #3: Offer Explicit Instructions for Sharing Different Types of Knowledge
Key Points
• Explain with specific instructions to the crowd both how knowledge integration should occur and types of knowledge to be shared.

120

UNIVERSITY OF CALIFORNIA, BERKELEY

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

Managing Crowds in Innovation Challenges

• Configure the technology to make sharing and integration of different types of knowledge easier to help the crowd access, highlight, and combine the different types of knowledge to develop integrative solutions.
In conventional innovation challenges, managers often focus on increasing the participation level and diversity of the crowd. While the level of participation may be important to the promotional success of an innovation challenge, having different types of knowledge to integrate appears to be particularly important for developing better solutions. Therefore, managers should monitor innovation challenges to ensure that these different types of knowledge are being shared. When noticeable knowledge gaps exist, managers may want to invite and incentivize additional participants to fill these gaps. Mangers may also want to moderate (or assign moderators) to encourage the crowd to fill the knowledge gaps.

TABLE 4. Characteristics of Innovation Challenges Encouraging Knowledge Integration
Crowd Management Focus
Guidelines for Crowd Behavior

Implementation
Knowledge Integration Instructions:
• Promote sharing, highlighting, and combining knowledge
• Emphasize that solutions are developed in an evolutionary fashion through natural discussions
• Ask crowd to highlight knowledge relevant to the innovation challenge no matter where it is posted in a thread—vote on both top-level posts and (more importantly) comments Specific Instructions for Sharing Different Types of
Knowledge:
• Emphasize that anyone can participate by adding knowledge of the following types—examples, facts, tradeoffs, seeds, and solutions

Dual Incentive Design

• Encourage crowd participants to engage in those knowledge integration actions and types they are comfortable with
Individual Output-Based Incentives
• Rewards for contributing to best solutions
Incentives to Encourage Process Behaviors that Facilitate
Integration
• Rewards for undertaking actions in the knowledge integration process (sharing, highlighting and combining)

Technical Design

• Design contribution mechanisms that emphasize diverse forms of knowledge needing to be contributed, not just “ideas”
• Ensure that both comments and top-level posts can receive votes, and that voting labels explain the basis for voting as
“useful knowledge for generating integrative solutions”
• Label knowledge sharing types to ensure crowd shares and attends to all types
• Allow (and require) the knowledge shared to be tagged based on the knowledge types for easy identification, retrieval and combination into integrative solutions

CALIFORNIA MANAGEMENT REVIEW

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

121

Managing Crowds in Innovation Challenges

Conclusion
The results presented provide indications that a crowd can be managed in a manner that increases the likelihood of developing better solutions. Instructions encouraging sharing of specific knowledge types help crowds to access and integrate a diverse range of knowledge. Dual incentives for the crowd encourage each of the actions in the knowledge integration process to be performed. It is thus possible to manage crowds to become active in the process of knowledge integration.
Table 4 summarizes the recommended crowd management practices resulting from this research.
The open innovation contexts studied in this research focused on the illdefined problems of a company such as new ideas for products and services. Of equal interest are contexts in which crowdsourcing is used to generate solutions to well-defined problems, such as algorithms. The suggestions presented above may be adaptable to such contexts, as knowledge integration is helpful for generating solutions for a range of problem types. The solutions identified by a crowd integrating knowledge—even with well-defined problems—may be even more efficient or more effective when integrated across multiple minds.
Companies have much to learn from crowds. Identifying how to harness the power of crowds is the next critical step for companies to differentiate themselves in the marketplace and sustain their competitive edge.

Notes
1. Crowdsourcing is defined as “the act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call. This can take the form of peer-production (when the job is performed collaboratively), but is also often undertaken by sole individuals. The crucial prerequisite is the use of the open call format and the large network of potential laborers.” Jeff
Howe, “The Rise of Crowdsourcing,” Wired Magazine, 14/6 (2006): 1-4.
2. Gartner Group, “Gartner Top Predictions 2014: Plan for a Disruptive, but Constructive
Future,” Gartner Report #G00257728, October 7, 2013.
3. L. Huston and N. Sakkab, “Connect and Develop: Inside Procter & Gamble’s New Model for
Innovation,” Harvard Business Review, 84/3 (March 2006): 58-66.; John Morgan and Richard
Wang, “Tournament for Ideas,” California Management Review, 52/2 (Winter 2010): 77.
4. D. Wright, “Thinking of Running an Open Innovation Contest? Think Again,” MIT Technology
Review, June 05, 2013; C. Rozwell, “Crowdsourcing,” in F. Karamouzis, “Hype Cycle for Application Services, 2013,” Gartner Report #G00248974, July 31, 2013.
5. I. Nonaka and N. Konno, “The Concept of ‘Ba’: Building a Foundation For Knowledge Creation,” California Management Review, 40/3 (Spring 1998): 40-54; I. Nonaka, “Redundant, Overlapping Organization: A Japanese Approach to Managing the Innovation Process,” California
Management Review, 32/3 (Spring 1990): 27-38; R.M. Grant, “Toward a Knowledge-Based Theory of the Firm,” Strategic Management Journal, 17 (Winter 1996): 109-122; B. Kogut and
U. Zander, “Knowledge of the Firm, Combinative Capabilities, and the Replication of Technology,” Organization Science, 3/3 (August 1992): 383-397.
6. H.W.J. Rittel and M.M. Webber, “Dilemmas in a General Theory of Planning,” Policy Sciences, 4/2
(June 1973): 155-169.
7. H. Tsoukas, “The Firm as a Distributed Knowledge System: A Constructionist Approach,”
Strategic Management Journal, 17 (Winter 1996): 11-25.
8. R.M. Grant, “Prospering in Dynamically-Competitive Environments: Organizational Capability as Knowledge Integration,” Organization Science, 7/4 (July/August 1996): 375-387; M. Alavi and A. Tiwana, “Knowledge Integration in Virtual Teams: The Potential Role of KMS,” Journal of the American Society for Information Science and Technology, 53/12 (October 2002): 1029-1037;

122

UNIVERSITY OF CALIFORNIA, BERKELEY

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

Managing Crowds in Innovation Challenges

9.
10.
11.

12.
13.

14.

15.

16.

17.

18.

J. Nahapiet and S. Ghoshal, “Social Capital, Intellectual Capital, and the Organizational
Advantage,” Academy of Management Review, 23/2 (April 1998): 242-266.
A. Majchrzak, P.H.B. More, and S. Faraj, “Transcending Knowledge Differences in CrossFunctional Teams,” Organization Science, 23/4 (July/August 2012): 951-970.
A. Majchrzak, C. Wagner, and D. Yates, “The Impact of Shaping on Knowledge Reuse for
Organizational Improvement with Wikis,” MIS Quarterly, 37/2 (June 2013): 99-104.
There is some research on innovation challenges that has encouraged knowledge integration through the creation of teams, where each team works independently of other teams and often offline (e.g., Top Coder contests, NASA contests, or with intermediaries such as Innocentive that allow challenge solvers to form ad hoc teams). However, subdividing the crowd into teams does not follow the process of knowledge integration since knowledge is not shared, highlighted, and combined among all participants in the innovation challenge. K.J. Boudreau,
N. Lacetera, and K.R. Lakhani, “Incentives and Problem Uncertainty in Innovation Contests:
An Empirical Analysis,” Management Science, 57/5 (May 2011): 843-863.
Boudreau, Lacetera, and Lakhani, op. cit.
J. Lampel and B. Ajay, “The Role of Status Seeking in Online Communities: Giving the Gift of
Experience,” Journal of Computer‐Mediated Communication, 12/2 (January 2007): 434-455;
S. Faraj, S.L. Jarvenpaa, and A. Majchrzak, “Knowledge Collaboration in Online Communities,”
Organization Science, 22/5 (September/October 2011): 1224-1239; M.M. Wasko and S. Faraj,
“It Is What One Does”: Why People Participate and Help Others in Electronic Communities of
Practice,” The Journal of Strategic Information Systems, 9/2-3 (September 2000): 155-173.
Each “top-level post” and “thread comment” was coded as whether it included an idea seed, defined as short actionable suggestions with prescriptions (e.g., “Develop a new line of toys that would stimulate interest in math and science at a very early age for young girls. Pink math and science toys?”). Each post was also coded for whether it contained other knowledge contributions: facts about the challenge (e.g., “children under five watch television about seven hours a day on average”); tradeoffs (e.g., “To switch from a B2B to B2C business model requires bypassing our service providers which means the company will simultaneously need to overcome the resistance from our service providers and build the channel to reach end users, which I think would be very difficult”); and examples (“I’m thinking of another company that I’ve seen doing what we’re thinking of doing”).
A. Tiwana and E.R. Mclean, “Expertise Integration and Creativity in Information Systems
Development,” Journal of Management Information Systems, 22/1 (Summer 2003): 13-43; Alavi and Tiwana, op. cit.; L.P. Robert Jr., A.R. Dennis, and M.K. Ahuja, “Social Capital and Knowledge Integration in Digitally Enabled Teams,” Information Systems Research, 19/3 (September
2008): 314-334.
This ratio of one-fourth of the participants actively posting with rest as lurkers is higher than in most other online innovation community projects, such as Wikipedia where the ratios are one-tenth. C. Wagner and A. Majchrzak, “Enabling Customer-Centricity Using Wikis and the Wiki Way,” Journal of Management Information Systems, 23/3 (Winter 2007): 17-43.
G. Kane, J. Johnson, and A. Majchrzak, “Emergent Lifecycle: The Tension between Knowledge Change and Knowledge Retention in Open Online Co-Production Communities,” Management Science (in press).
A. Majchrzak and A. Malhotra, “Towards an Information Systems Perspective and Research
Agenda on Crowdsourcing for Innovation,” The Journal of Strategic Information Systems, 22/4
(December 2013): 257-268.

California Management Review, Vol. 56, No. 4, pp. 103–123. ISSN 0008-1256, eISSN 2162-8564. © 2014 by The Regents of the University of California. All rights reserved. Request permission to photocopy or reproduce article content at the University of California Press’s Rights and Permissions website at http://www.ucpressjournals.com/reprintinfo.asp. DOI: 10.1525/cmr.2014.56.4.103.

CALIFORNIA MANAGEMENT REVIEW

VOL. 56, NO. 4

SUMMER 2014

CMR.BERKELEY.EDU

123

Copyright of California Management Review is the property of California Management
Review and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.