Citizen Science as an Approach for Overcoming Insufficient Monitoring and Inadequate Stakeholder Buy-in in Adaptive Management: Criteria and Evidence

Erendira Aceves Buenoa, Adeyemi S. Adeleyea, Darcy Bradleya, William Brandta, Patrick Callerya, Marina Ferauda, Kendra Garnera, Rebecca Gentrya, Yuxiong Huanga, Ian McCullougha, Isaac Pearlmana, Sara Sutherlanda, Whitney Wilkinsona, Yi Yanga, Trevor Zinka, Sarah Andersona,b, Christina Taguea

a Bren School of Environmental Science & Management, University of California, 2400 Bren Hall, Santa Barbara, CA 93106

b Corresponding author: sanderson@bren.ucsb.edu; (805) 893-5886

Authors’ contribution

ABSTRACT

Adaptive management is broadly recognized as critical for managing natural resources, yet in practice it often fails to achieve intended results for two main reasons: insufficient monitoring and inadequate stakeholder buy-in. Citizen science is gaining momentum as an approach that can inform natural resource management and has some promise for solving the problems faced by adaptive management. Based on adaptive management literature, we developed a set of criteria for successfully addressing monitoring and stakeholder related failures in adaptive management and then used these criteria to evaluate 83 citizen science case studies from peer reviewed literature. The results suggest that citizen science can be a cost-effective method to collect essential monitoring information and can also produce the high levels of citizen engagement that are vital to the adaptive management learning process. The analysis also provides a set of recommendations for citizen science program design that addresses spatial and temporal scale, data quality, costs, and effective incentives to facilitate participation and integration of findings into adaptive management.

KEYWORDS

Citizen science; Community based monitoring; Crowd science; Public Participation in Scientific Research; Adaptive management; Natural resource management; Environmental science and management

INTRODUCTION

Over the past 35 years, adaptive management (Walters and Hilborn, 1976; Holling, 1978) has grown in popularity and is frequently recommended as a strategy for natural resource management (Johnson, 1999; Doremus, 2010; Keith et al., 2011; Williams, 2011; Rist et al., 2013). In spite of numerous successes, adaptive management has often failed to deliver on its promise (Walters, 2007; Keith et al., 2011). In many cases, these failures have resulted not from inherent flaws in the concept of adaptive management, but rather in its incorrect, incomplete or inappropriate implementation (Gunderson, 1999; Gregory et al., 2006; Reever Morghan et al., 2006; Walters, 2007). Reviews of adaptive management case studies have identified a number of different reasons for failures. Two of the most often-cited reasons for lack of success are inadequate system monitoring and low stakeholder buy-in. System monitoring is a core component of adaptive management and provides the data used to evaluate the relative success of different management strategies. Monitoring, however, is often limited by available financial, technical or human resources (Walters, 2007; Allen and Gunderson, 2011; Keith et al., 2011). Adaptive management also relies on stakeholder acceptance of different management strategies. (Meffe et al., 2010). Lack of transparency and information asymmetry between managers and stakeholders can present a barrier to successful collaborative adaptive management programs (Plummer and Armitage, 2007; Monroe et al., 2013). Because adaptive management typically involves high levels of uncertainty, consistent stakeholder engagement throughout the management process helps to prevent conflicts arising from unexpected results (Allen and Gunderson, 2011). In this paper we explore whether the integration of citizen science into adaptive management can address these two core challenges for adaptive management: inadequate system monitoring and low stakeholder buy-in.

In recent years, citizen science, where a non-expert “collects and/or processes data as part of a scientific enquiry” (Silvertown, 2009), has been a valuable addition to natural resource management (Metzger and Lendvay, 2006; Pittman and Dorcas, 2006; Lye et al., 2012; Gura, 2013; Silvertown et al., 2013). Since citizen science, also called Public Participation in Scientific Research (PPSR;Shirk et al., 2012), provides monitoring data and offers a mechanism for engaging the public, it seems likely that it could contribute to adaptive management. Environmental science is already one of the areas most commonly engaging citizen scientists and there are well known examples of citizens successfully monitoring environmental variables (Silvertown, 2009). The National Biodiversity Network in the UK, for example, engages tens of thousands of people to compile observations of species. Researchers at Arizona State University recently launched the Ventus Project, a program to gather comprehensive information about global locations of power plants and how much carbon dioxide they are emitting using citizen scientists. The success of several programs have relied heavily on the contributions of citizen scientists include Galaxy Zoo, the Audobon Society’s Christmas Bird Count, and NASA’s Quantum Moves. Dickinson et al. (2010) review several examples of citizen science used to provide data on environmental topics ranging from infectious diseases to phenology to habitat loss.

In addition to their most common involvement in data collection, stakeholders can be involved in design and implementation, analysis of data, and dissemination of results. By directly engaging stakeholders in both programmatic design and implementation, citizen science projects may engage them through the acquisition of new knowledge or environmental stewardship. Since citizen scientists are likely to share information they have helped to collect, this also enhances information flow and the development of new or existing relationships (Reed, 2008). Participation in citizen science programs has been shown to empower citizens by increasing participants’ scientific literacy (Price and Lee, 2013) and their likelihood of becoming an environmental advocate in the long-term (Ryan et al. 2001). While these scholars and others who have reviewed the use of citizen science identify limitations and gaps, they generally conclude that citizen science can make a positive contribution to natural resource management.

The use of citizen science specifically within the context of adaptive management of natural resources is a relatively new concept and has not been extensively evaluated. Cooper et al. (2007), for example, propose a framework for using citizen science for adaptive management of residential ecosystems, drawing on the successful experience of the Cornell Lab of Ornithology with citizen science volunteers. Our study evaluates the use of citizen science in a broad range of previous environmental analysis and uses these studies to assess whether citizen science might address core challenges in adaptive management. In this paper we examine whether and under what conditions citizen science might contribute to adaptive management. Specifically we ask whether citizen science can reduce failure due to inadequate monitoring or insufficient stakeholder buy-in. To answer this question we begin by identifying the specific characteristics of system monitoring and stakeholder engagement that previous literature has indicated are important for successful adaptive management. We posit this list of characteristics as a set of criteria that must be met if citizen science is to address the two common barriers to successful adaptive management; inadequate monitoring and low stakeholder buy-in. We then review 83 case studies of the use of citizen science within the peer reviewed literature to assess how often and under what conditions they meet these criteria.

From this meta-analysis, we also identify salient examples of successful (and unsuccessful) citizen science programs. We use these examples to synthesize quantitative and qualitative lessons that can guide the effective use of citizen science for adaptive management. We conclude by providing a set of recommendations on the design, spatial and temporal scale, costs, and incentives of citizen science to facilitate its effective integration into adaptive management.

Criteria for Successful Adaptive Management

In this section we discuss two often cited reasons for failure of adaptive management, which we classify as failures due to inadequate system monitoring and failures of stakeholder buy-in. We draw on previous critiques of adaptive management to determine when, why, and how monitoring and stakeholder engagement present challenges to adaptive management. We use this review to develop a set of criteria that, if met, would lead to the successful use of citizen science to overcome either inadequate system monitoring or failures of stakeholder buy-in. Although previous authors have identified broad criteria for adaptive management to be successful in general (Gregory et al., 2006), here we focus on specific criteria that must be met in order to overcome the particular challenges of monitoring and stakeholder engagement. We then use these criteria to evaluate citizen science case studies and determine whether there is evidence that citizen science is likely to contribute to improved adaptive management. These criteria are translated into a list of questions asked of each citizen science paper in our review (discussed in more detail below and in Table 1).

Monitoring Criterion 1 (MC1): Monitoring must take place.

The goal of adaptive management is to incorporate learning over time to reduce uncertainty and improve resource management (Lyons et al., 2008; Doremus, 2010), and data from monitoring is integral to learning. Over the past four decades, projects that strictly met the criteria of adaptive management were very few (Westgate et al., 2013) and ineffective system monitoring is often identified as a major contributor to failures in adaptive management systems (Allen and Gunderson, 2011). In fact, gaining sufficient information to inform the management process is a necessary first step for successful adaptive management (Doremus, 2010). As Doremus (2010) puts it, “Only if learning is feasible does it make sense to worry about whether managers want to learn, can be forced to learn, or can use knowledge they acquire.” Effective monitoring programs are often credited with contributing to the success of adaptive management. The US Fish and Wildlife Service together with the Canadian Wildlife Service, for instance, attribute the success of adaptive management of waterfowl to monitoring programs at the state, provincial, and private-partner levels (Nichols et al., 2007).

Monitoring Criterion 2 (MC2): Data collected by monitoring programs must be relevant to management actions, quantitative, and pass rigorous quality and consistency controls.

Monitoring for management objectives is more than omnibus surveillance or mere information gathering (Lee, 1999; Nichols and Williams, 2006). Monitoring programs should be intentionally designed to reduce uncertainty about the effectiveness of management actions (Allen and Gunderson, 2011). Often, however, monitoring data, which “must be sensitive to management actions” (Nichols and Williams, 2006), are not collected in ways that directly relate to management outcomes and therefore do not reduce uncertainty. Scientists who do primary data collection and monitoring may not understand the overall scope of management objectives, and therefore may fail to “recognize the need to provide information that can be directly used by decision makers” (Gregory et al., 2006), or worse, may “hijack” the project for research interests (Walters, 1997). The data monitoring program must be designed with the explicit objective of providing feedback on management experiments, thereby informing management decisions (Ringold et al., 1996).

Monitoring data must also be quantitative and well-defined (Ringold et al., 1996) and confounding variables kept to a minimum. If monitoring data are of poor quality, the results can be catastrophic. Studies have argued that the collapse of the Newfoundland cod fishery was due at least in part to poor monitoring of fish populations (Walters, 2007). Wildlife monitoring surveys are particularly prone to errors including incomplete detection, non-detection, and misidentification due to observer bias (see, for example, Lotz and Allen, 2007; Shea et al., 2011; Farmer et al., 2012). Therefore, systems must be in place to ensure that data meet minimum quality and consistency standards and decision makers must be made aware of how any biases or shortcomings in monitoring data might affect uncertainty in decision outcomes.

Monitoring Criterion 3 (MC3): Monitoring costs must be kept below the benefits of the management objective.

A third critical reason monitoring fails is the sheer expense of monitoring programs (Lee, 1999; Allen and Gunderson, 2011). In several cases of fisheries management, for instance, the cost to monitor fish stocks can approach or even outweigh the benefits to the users (Walters, 2007). There are many factors contributing to the high cost of monitoring, including the need for repetition over large scales (Walters, 2007; Keith et al., 2011), the fact that monitoring is traditionally done by in-house staff or over-qualified researchers (Walters, 2007), or the misallocation of monitoring resources to components of a system that are ultimately unimportant or unresponsive to management options (Keith et al., 2011). These costs can either be direct costs of the data collection and analysis itself or opportunity costs of using a resource as an experiment (Allen and Gunderson, 2011). Therefore, the costs of monitoring must be accounted for when designing an adaptive management solution and the monitoring program must be designed to ensure that the costs do not outweigh the benefits of the overall adaptive management program.

Monitoring Criterion 4 (MC4): Monitoring programs must be designed to accommodate the appropriate spatial and temporal scales.

The large temporal and spatial scales often required for adaptive management can pose difficulties for monitoring (Devictor et al., 2010), Most natural resource management decisions are made at relatively short quarterly or annual time scales; however, acquiring usable data from experiments run on natural systems generally requires much longer times (Doremus, 2010) and iterative approaches and replication of experiments can be difficult (Allen and Gunderson, 2011). For example, in the Chesapeake Bay, water quality scientists believe it will take at least nine years to recognize how changes in agricultural practices affect nutrient runoff (Doremus, 2010). Long timeframes raise the possibility of funding cuts, policy changes, confounding natural events such as fires or floods, and personnel changes (Westgate et al., 2013).

Aside from increased monetary costs, large spatial scales require buy-in from more stakeholders and participation from more diverse regulatory agencies, which makes uniform data collection more difficult and reduces the control the managers have over policy experiments (Gregory et al., 2006; Allen and Gunderson, 2011). Additionally, small-scale projects require very different monitoring solutions from large-scale projects; adaptive management practitioners must carefully consider the scales involved when designing the monitoring program (Ringold et al., 1996) and must be cognizant that large spatial scales and long timelines present special challenges (Nichols and Williams, 2006).

Stakeholder Criterion 1 (SC1): Community stakeholders must be identified and engaged in the management process.

Adaptive management requires that those directly affected have the capacity and the motivation to accept management changes (Susskind et al., 2012). In a study of 105 cases of ecosystem-based public resource management, the presence of dedicated stakeholders was the most important factor in enabling successful program outcomes (Yaffee, 2002). Identification and inclusion of diverse stakeholder groups can enable a deeper base of knowledge from which to inform natural resource management (Reed et al., 2009). This is especially true for adaptive management, where equitable and consistent stakeholder engagement throughout the management process may help to mitigate conflicts arising from unexpected results (Allen and Gunderson, 2011).

Formal definitions of stakeholder vary widely depending on context. All participants in a given study may be considered stakeholders, given that society as a whole is impacted by environmental issues and any individual’s involvement in monitoring may contribute to broad societal acceptance of management practices. However, local stakeholders who have a direct economic or health stake in the management outcomes may be particularly critical to engage in successful adaptive management (Fernandez-Gimenez et al., 2008). In order to examine the engagement of community stakeholders with a direct linkage to the management issue being studied, we classify participants as either community members or volunteers. Community members are defined as those with a direct stake in management outcomes. Volunteers, on the other hand, participate in citizen science even though they have no direct economic or health interest in the resource being managed. We acknowledge that this classification is loosely defined, but inter-coder reliability tests (detailed below) on the question of whether participants were volunteers or community stakeholders indicate 100% agreement.

Stakeholder Criterion 2 (SC2): To facilitate engagement of citizens, managers must provide appropriate motivation and incentives for participation.

Mere involvement of stakeholders in adaptive management frameworks is not sufficient to ensure effective resource management; it also requires tools and incentives that encourage and support engagement in the process (Susskind et al., 2012). For stakeholders with very strong economic or social ties to a particular project, the motivations for active participation may be self-evident. In other cases, additional incentives may be required. Many natural resources offer a sense of community to individuals and interest groups with long-term relationships to the resource (Fleming and Love, 2012), and fostering effective collaboration between stakeholders and managers can improve the level of engagement that is vital to success of adaptive management programs (Sale et al., 2005). Simply providing a forum for stakeholders to express views allows participants to feel represented and gain information and knowledge through the process of adaptive management (Linkov et al., 2006).

Stakeholder Criterion 3 (SC3): Decision-makers must be accountable to stakeholders while maintaining focus on the adaptive management goal.

Engagement of stakeholders in adaptive management requires an expectation that the interests and concerns of stakeholders will be taken into consideration by decision-makers. Regular interactions among natural resource managers and stakeholders, whether remote or in person, are essential for the long-term success of a project (McKenzie et al., 2000; McKenzie et al., 2001; Danielsen et al., 2005). Allowing stakeholders to influence the questions asked and the scientific outputs produced can instill a sense of ownership and responsibility for the success of the project (Reed, 2008; Prell et al., 2009). When this fails to occur, indifference, distrust and tension can undermine adaptive management (Linkov et al., 2006).

However, accountability can be a double-edged sword and requires balance. By yielding unnecessary control to activist stakeholders, managers may unwittingly allow bureaucracy to overwhelm flexibility in the iterative processes of adaptive management (Habron, 2003). The more that stakeholders are allowed to steer the course of science, the more accountability can lead to erroneous learning and poor decisions (Lee, 1999). While ensuring accountability to stakeholder interest, managers must maintain control of the scientific mission.

Table 1 summarizes the criteria identified from the literature on adaptive management that we use to evaluate the potential for citizen science to address monitoring and stakeholder involvement challenges in adaptive management. Of course, even if a project meets these criteria, there are other factors outside the realms of monitoring and stakeholder engagement that may limit its effectiveness.

METHODS

In order to determine whether existing uses of citizen science offer evidence that it can meet the monitoring and stakeholder engagement criteria established above, we analyzed 83 peer reviewed articles discussing the use of citizen science in specific natural resource management settings. We developed a series of questions (details provided in Appendix A) that we answered for each article. These questions were designed to determine whether the use of citizen science provides the necessary criteria for successful adaptive management as outlined in Table 1. The coders were graduate students from across environmental science, social science, and management.

Relevant studies were identified through computer searches of the Web of Knowledge using the keywords ‘“monitor” and “citizen science” or “community based monitoring”. Search results were narrowed to consider only papers published after 1990 in research areas related to environmental science, ecology, conservation, and natural resource management – yielding a preliminary list of 186 papers. Searches for “crowd science”, “crowd-source science” and “network science” yielded no additional relevant articles. Review articles were eliminated from the analysis, as were articles that did not describe a specific citizen science program, and articles without enough information to assess whether they met the monitoring and stakeholder criteria, resulting in a final sample of 83 papers. These papers are listed in Appendix B.

Intercoder Reliability Analysis

An intercoder reliability (ICR) analysis was performed, in which a subset of 13 out of the 83 total papers was randomly selected for coder agreement. Percent agreement, although popular due to its easy computation, is not an adequate measure of reliability by itself as it may overestimate agreement by failing to account for chance agreement (Lombard et al., 2002). For this reason, the Fleiss’s K coefficient, a conservative index, was also computed, following the procedures described in Fleiss (1971) and Landis and Koch (1977).

Guidelines regarding the interpretation of the values of different ICR indices exist, but these are not universally accepted. Some suggest that coefficients of at least 0.80 are acceptable and that coefficients of 0.70 are appropriate for exploratory studies (Lombard et al., 2002). Landis and Koch (1977) suggest that a Fleiss’s K coefficient in the 0.61-0.80 range shows substantial agreement, and in the 0.41-0.60 range shows moderate agreement. For this study, coding reliability for a given category is considered (i) substantial, if Fleiss’s K is equal to or larger than 0.61 and percent agreement is equal or greater than 85% (this amounts to no more than 2 disagreements on the 13 papers included in the ICR analysis), and (ii) moderately reliable if Fleiss’s K is equal or greater than 0.41 and less than 0.61 and percent agreement is at least 69% (no more than 4 disagreements).

Most categories were reliably coded, showing either substantial (56%) or moderate (37%) agreement. Only two categories were not reliably coded; these were “QB3: Who was involved in the design of the monitoring?” (which was replaced by “QO3: Were the managers directly involved in the design of the study?”) and “QSe4: How do citizen scientists and managers interact?” (see Appendix A for a list of all categories). Three categories had high skewness in answers which has been shown to lead to low Fleiss’s K values. We considered these three categories moderately reliable. These were “QD1: Are quantitative data (categorical or numeric) collected?”, “QD4: Are QA/QC metrics for the citizen science collected data discussed?” and “QSd1: What benefits do the citizen scientists receive?” A summary of the ICR results by question is presented in Appendix C.

The data collected through the review and systematic coding of citizen science case studies were used to evaluate how often and under what conditions citizen science met our criteria for monitoring and stakeholder engagement in adaptive management. We discuss the findings for each criterion below and note in brackets the code corresponding to the exact wording of the question available in Appendix A. We supported this assessment with a multivariate regression analysis to determine which criteria were correlated with the dependent variable of particular interest: whether data from citizen science were actually used in resource management [QO2]. Since collection of citizen science data for adaptive management purposes does not necessitate that managers will use the data for management decisions, we argue that examples where citizen science was reported as useful for management are likely to be good indicators of the potential for successful application in an adaptive management context. Details on the multivariate regression are provided in Appendix D and results are reported in the text throughout.

RESULTS

MC1: Citizen science and monitoring

Not surprisingly, it is clear that citizen science can be used for monitoring of environmental resources: all 83 of the case studies used citizen science for environmental monitoring, in some cases with great success. For example, in South Africa the Protea Atlas Project recruited and trained over 400 citizens who successfully mapped 377 species of Proteaceae during a 10 year period, ultimately yielding one of the largest plant databases in the world with over 250,000 records. Monitoring data collected by the Protea Atlas Project citizen scientists have been used for conservation planning, to determine threatened species, and to apply for International Union for Conservation of Nature (IUCN) Red Listing status (Silvertown, 2009). Similarly, citizen scientists across the UK collected data on 1,022 bumblebee nests from 2007 to 2009 throughout the United Kingdom, which would have been very expensive and time-consuming for professional biologists. The data provided the first quantitative evidence of potential declines in UK’s commonest bumblebee, Bombus pascuorum (Lye et al., 2012). Community members of Bayview Hunters Point (San Francisco, California) were trained to carry out water quality measurements at Yosemite Slough. The citizen scientists successfully gathered baseline data for watershed restoration and identified potential pollution sources and health hazards (Metzger and Lendvay, 2006). In the case studies reviewed, citizen science was used to monitor a wide variety of environmental resources – from sharks (Davies and White, 2012) to ladybugs (Gardiner et al., 2012) to soil quality (Bone et al., 2012) to biodiversity (Oldekop et al., 2011).

MC2: Data quality and relevance to management actions

We found that citizen science monitoring was often relevant to management decisions: of the 83 studies analyzed, 46% reported clearly stated management objectives [QO1] and 41% reported that data was used to make management decisions [QO2]. For instance, Van Rijsoort and Zhang ( 2005) describe citizen science biodiversity conservation monitoring efforts in the forests of Yunnan, China, which led to the prohibition of tree cutting for fruit provision, restrictions on wild forage for livestock, and the enactment of a management plan for medicinal orchids. The multivariate analysis (Appendix D) also showed that those citizen science programs designed with the explicit goal of informing management [QO1] were significantly more likely to use data collected for management. Although managers were directly involved in the study design in 39% of the studies analyzed [QO3], participation by managers in study design did not significantly predict whether or not data collected were ultimately used for management (p>0.10, Appendix D). To ensure that useful and actionable data are collected, therefore, it is important to incorporate management objectives in study design.

Of the citizen science monitoring efforts studied here, 90% collected quantitative data [QD1] and 81% included quality assurance and control (QA/QC) mechanisms [QD4] associated with data collected by citizen scientists. Relative to studies that collected only quantitative data, those that collected qualitative data (either alone or in addition to quantitative data) were less likely to use the data for management (Appendix D). Among the studies mentioning the use of QA/QC, only 11% reported having no data accuracy issues [QD5]. However further analysis reveals that the accuracy and reliability issues identified with citizen science data often do not affect the overall utility of the data. Specifically, among the 57 studies in which citizen science data were identified as being problematic, 59% found that the problems were minor, while 39% found them critical but fixable or workable [QD6]. Onlyi one study reported that poor quality made data unusable. We note however that publication bias may lead to an under-reporting of studies with poor quality data. Examples of citizen science producing data statistically comparable to professionals were common (Crall et al., 2010; Finn et al., 2010; Hoyer et al., 2012). Studies with QA/QC mechanisms were no more likely to use the data for management than those without them (Appendix D). Like Danielsen et al. (2005), we conclude that citizen science programs can provide robust monitoring data for use in adaptive management as long as systems are in place to ensure that data meet minimum quality and consistency standards and decision makers are made aware of any biases or shortcomings.

MC3: Cost effectiveness

The studies reviewed provide several examples where citizen science programs offered a promising solution to cost-effective monitoring through the regular use of citizens for data gathering. For example, one study found that the costs of monitoring forest habitat were as much as two orders of magnitude lower when using citizens compared to professionals (US$0.01 - $0.04 versus US$1.88 per hectare per year; Holck, 2008). In another example, Levrel et al. (2010) describe a French citizen science program monitoring bird and butterfly populations that resulted in net savings between €0.7-4.4 million per year, as compared to the expected costs of monitoring by government employees. Hamilton et al. (2012) present an example of fisheries monitoring where use of citizen science would have been too expensive for the community to enact without external support. In this case, estimated annual costs of monitoring a fish spawning aggregation (FSA) were deemed prohibitive due to salaries of organizers, boat hire, SCUBA equipment, and insurance. Hamilton and his colleagues suggest that the viability of future FSA monitoring requires external financial support from NGOs and national government agencies.

The studies reviewed also highlight that there are different ways to provide training that can impact project costs. In some cases the requisite skills could be learned from peers, for example by joining a diving club (Pattengill-Semmens and Semmens, 2003) or hiking group (Brook et al., 2009). This method represents the lowest-cost method of training; however it requires significant commitment by participants to engage in these activities. In 58% of cases we reviewed, the organizer of the citizen science monitoring program provided training [QM2], and the length of training varied from 15 minutes (Fitzpatrick et al., 2009) to 30 days (Becker et al., 2005). Because training usually requires facilities, equipment, and salary for the trainer, increasing training time generally accrues higher project costs. Costs also depend on the method of administering training. Often training takes the form of an in-person session led by a project coordinator (Humber et al., 2011; Jordan et al., 2011), but lower cost training may be achieved by allocating information packets (Crewe et al., 2005) or utilizing online resources for broad dissemination of training information (Ellul et al., 2011; Lye et al., 2012).

In summary, while there are clear examples where citizen science improves cost effectiveness relative to traditional scientific monitoring involvement professionals, relative cost-effectiveness depends on characteristics of the study. Factors such as the cost associated with professional monitoring, the nature and extent of training required, the scale of program, and monitoring duration all influence the costs of citizen science relative to monitoring by experts. Further, in this study, cost effectiveness was assessed using monitoring cost. However, the use of citizens in the monitoring process may reduce the cost of achieving community engagement. Thus, full accounting of the cost-effectiveness citizen science in adaptive management would have to consider costs and benefits beyond those associated with data collection. The studies reviewed here did not include sufficient information to evaluate these non-monitoring related benefits.

MC4: Temporal and spatial scales

The duration of citizen science monitoring programs we reviewed range from as short as a few hours (Leopold et al., 2009) to over 40 years (Szabo et al., 2010; Brunsdon and Comber, 2012). While we found more case studies at the regional scale (57%), citizen science monitoring has been utilized for both global (2%) and local projects (41%) [QD3] and we found no difference in the likelihood of using the data for management across spatial scales (Appendix D). Some systems, such as restoration efforts in the Everglades, the Colorado River, the Columbia River, and the Platte River involve temporal or spatial scales that simply do not allow for iterative approaches and replication of experiments (Allen and Gunderson, 2011). However given the wide range of spatial and temporal scales found in our review, it is clear that citizen monitoring can be applied across many temporal and spatial scales (see Appendix A).

SC1: Community stakeholders must be clearly identified and engaged.

Citizen science can address the problem of poor stakeholder engagement in adaptive management only to the degree that stakeholders are used to conduct citizen science. This requires that natural resource managers identify these stakeholders and recruit them for involvement in a citizen science program and then ultimately continue to engage them in the process. If we use the very broad definition of stakeholder as discussed above (any individual or group with social, psychological, or economic interest in the resource being managed), then almost by definition all of our citizen science examples involved stakeholders.

Narrowing our definition of stakeholder to community members with a direct economic or health stake in the management outcome, we found that 72% of the 83 studies reviewed used volunteers, while 28% used community members [QSe5]. We refer to volunteers as individuals who self-select into a citizen science program for ostensibly subsidiary benefits, such as recreational interest.

Programs using volunteers generally succeed by engaging volunteers’ interests and offering relevant training or experience. For example, the Cornell Lab of Ornithology has engaged thousands of citizen science volunteers over multiple decades, furthering scientific understanding of bird species while providing educational opportunities for the public (Bonney et al., 2009b).

When citizen scientists were also community members, successful programs often involved specific activities to build trust between researchers and the communities around a common set of goals and objectives. For example, researchers of a marine turtle fishery in Madagascar worked together with community elders to design a monitoring program and determine appropriate incentives for participants (Humber et al., 2011). In a study of invasive agricultural pests in Ecuador, researchers engaged local farmers through community forums to develop common agreement on the key objectives of the monitoring program (Dangles et al., 2010). That more than a quarter of the citizen science studies involved community members suggests that citizen science can be used to engage local community members, whose acquiescence to management changes may be crucial.

SC2: Managers must provide appropriate motivations and incentives for participation.

Understanding what can be used to encourage participation of citizen scientists in resource monitoring is critical in integrating citizen science into adaptive management. We encountered five ways to engage citizen scientists in natural resource management: sense of place/social capital, tools and technology, action, knowledge and economics [QSd1]. Some of these (sense of place, action, and knowledge) are intrinsic motivators, while others are incentives that can be provided by managers. Sense of place serves as a motivator when citizen scientists feel culturally or aesthetically connected to the local community in which they work and/or live as a result of participation. Tools and technologies, such as integrated mobile phone applications, cameras, global positioning systems, and innovative web platforms, helped to incentivize participation. Innovative web platforms, for example, have been used to encourage citizen scientists to contribute information and interact with professional scientists (e.g., eBird; Jetz et al., 2012)). Action refers to the motivation implicit in helping interested or concerned citizens to feel that that they can make personal contribution to more effective resource management. Potential for knowledge gain can motivate citizen scientists to participate in resource monitoring. Finally, economic incentives include different forms of compensation (e.g., supply or transportation costs for resource monitoring or direct payments).

Of the 83 papers we reviewed, knowledge was the most frequently mentioned motivator (75%), whereas sense of place (49%), action (29%), tools and technology (25%) and economic incentives (22%) were less frequently encountered. Forty-five papers (54%) mentioned one or more incentive or motivator. Six projects (7%) used all five means of engagement, including community-based water quality assessment in the San Francisco Bay (Metzger and Lendvay, 2006) and invasive pest management in the Ecuadorian Andes (Dangles et al., 2010). The use of sense of place, technology, and action to encourage participants was associated with a higher likelihood of using the data for management in the multivariate analysis, whereas the use of knowledge attainment to motivate participants was negatively correlated with the use of data for management. The presence of economic incentives was not significantly correlated with the use of the data for management (see Appendix D).

SC3: Managers must be accountable to stakeholder interest.

To the extent that results are communicated to stakeholders, including citizen scientists in the adaptive management process provides legitimacy to management decisions by increasing transparency (Turnhout et al., 2010; Reed, 2008). 39% of our case studies reported that direct feedback of results was provided to participants [QSi1], with 66% of those including an opportunity for participants to respond to the communicated results [QSi2]. Those that offered explicit communication of the results were more likely to use the data for management, although we cannot know whether the communication was required because the data were to be used for management (Appendix D). Providing opportunities for participants to respond to the results did not increase the likelihood of using the data for management.

DISCUSSION & RECOMMENDATIONS FOR EMPLOYING CITIZEN SCIENCE IN ADAPTIVE MANAGEMENT

Our meta-analysis found that citizen science frequently addressed the monitoring and engagement criteria required to resolve some of the shortcomings of adaptive management. These results suggest that citizen science may indeed be an effective strategy for overcoming barriers to adaptive management related to inadequate monitoring and low stakeholder involvement. Of course, these examples do not guarantee the success of citizen science in a particular adaptive management context; we acknowledge the possibility of publication bias in our sample, where unsuccessful cases of citizen science used for adaptive management are less likely to be published. Further, we found examples where the costs of monitoring, the length of the monitoring program, lack of quality assurance and quality control for monitoring data, or lack of stakeholder engagement incentives may provide major obstacles to the success of citizen science initiatives in serving the needs of adaptive management. We therefore discuss these and provide recommendations to curtail the potential problems associated with each.

Costs

In aggregate, the literature reviewed suggests that citizen science can be a cost-effective method of monitoring for adaptive management. However, there were also exceptions (e.g., Hamilton et al., 2012). A synthesis of cost-effective and cost-ineffective examples reveals ways in which costs can be controlled to ensure that a citizen science project results in net benefits. In general, projects must be designed with full consideration of anticipated costs. Some articles described cost comparisons between using citizen scientists and other forms of monitoring (Levrel et al., 2010; Hamilton et al., 2012), which suggests that preemptive cost analysis is both feasible and highly valuable to the design of a project. Additionally, as we note above, training can affect costs, with in-person training and longer duration training costing more. Although it is often difficult to quantify benefits since these tend to be qualitative and long-term, to the extent possible expected benefits should be compared to anticipated costs. We again note that some of the benefits of citizen science may be to reduce costs associated with stakeholder engagement but we were not able to evaluate this in our review.

Length of monitoring program

While we found that citizen science monitoring programs can be successful at both short and long temporal scales, those projects collecting longer-term data were more likely to use volunteers than community members. This suggests a possible tradeoff between community member engagement and collection of the long-term data necessary for adaptive management. Since long-term monitoring is usually expensive, citizen-based resource monitoring programs are unlikely to persist without continuous outside support or strong engagement of citizen scientists. As an example, Becker et al. (2005) describe the fog and bird monitoring efforts made by a community in western Ecuador where, despite the project’s general success, its continuation faced challenges due to lack of funding sources. We therefore recommend that programs requiring monitoring duration longer than one year include provisions to engage citizen scientists to assist program persistence and that they make attempts to include community members, rather than only volunteers, among the citizen scientists.

Quality assurance and quality control

Data quality issues are not specific to citizen science monitoring programs (Dickinson et al., 2010); experienced scientists make data errors as well (Foster-Smith and Evans, 2003; Crall et al., 2011). Nonetheless, the accuracy and reliability of citizen science data is frequently questioned (Danielsen et al., 2005; Fitzpatrick et al., 2009; Dickinson et al., 2010; Gardiner et al., 2012). Given that most citizen scientists lack professional training, it is not surprising that problems of various degrees exist in many citizen science data collection efforts (Danielsen et al., 2005; Dickinson et al., 2010). This review, however, found that data from citizen science monitoring initiatives are largely valuable and that if errors exist they can generally be fixed or accounted for. Training appears to be the key contributor to enhancing citizen scientist performance (Miller and De'ath, 1996; Au et al., 2000; Silvertown et al., 2013; Sunde and Jessen, 2013). For example, expert-quality monitoring data can be produced by secondary school children given consistent and thorough training via extensive pre-monitoring classroom work (Cox et al., 2012). After appropriate training, high school students were able to collect water-monitoring data comparable to that of experts (Au et al., 2000). Other measures that have been shown to enhance citizen science data quality include the use of standardized protocols and provision of adequate resources for equipment (Sharpe and Conrad, 2006), as well as having program veterans accompany novice citizen scientists (Downs, 2005).

Data validation processes are also of paramount importance to ensure monitoring programs produce quality data. Quality control can be achieved through close communication and collaboration among citizen scientists, managers, and experts, as each party serves to keep the others in check (Bois et al., 2011). For example, the Invasive Plant Atlas of New England (IPANE) requires that experts verify questionable identifications as a form of quality control for data collected by trained citizen scientists. In another example, software applications (eBird) were developed to foster rapid data collection and review, including quality assurance checks via electronic communication with participants to validate questionable observations (Dickinson et al., 2010; Munson et al., 2010).

Stakeholder engagement incentives

If citizen science is going to promote stakeholder engagement in adaptive management, particularly long term engagement, incentives need to be an integral part of the program. One of the interesting findings from this review is summarized in Figure 1 as STAKE (Sense of place, Tools and technology, Action, Knowledge and Economic benefits), a five-point framework summarizing the different incentives that can motivate citizen scientists to become engaged stakeholders in natural resource management. This is encouraging and suggests that there is some flexibility in how citizen science can be used to provide motivation within the context of stakeholder engagement in adaptive management. Given the findings that the type of motivation affects whether the data from citizen science is used in management, further study of what motivates stakeholder engagement is likely to be fruitful.

CONCLUSION

Engaged citizen scientists can address and correct the shortcomings of adaptive management

Citizen science can be a critical, missing piece in solving the adaptive management puzzle. Motivated citizen scientists have the ability to rectify two key shortcomings in adaptive management: the need for adequate monitoring and stakeholder engagement. Our review of peer reviewed examples of the use of citizen science found that a well-designed citizen science program with engaged participants can contribute to effective adaptive management due to its ability to provide cost-effective, quality controlled information, gathered specifically for management, and at appropriate spatial and temporal scales. In addition to providing data for management, successful citizen science programs can engage community stakeholders to develop a greater sense of stewardship over natural resources and become personally invested in sustainable resource management (e.g., Cornwell and Campbell, 2012). Future work could consider whether the way that citizens are engaged (contributory, collaborative, or co-development; Bonnery, et al., 2009a) affects the likelihood that citizen science is useful in adaptive management or to achieve information sharing, scientific literacy, and increased involvement. High levels of participation and engagement can be encouraged though a wide range of both extrinsic incentives and non-economic motivators. However, engaged citizen science is not a catch-all solution and addresses only part of the adaptive management puzzle. Resource managers are still ultimately responsible for designing monitoring programs with management objectives in mind, establishing quality controls, and effectively utilizing data collected by citizen scientists for natural resource management.

ACKNOWLEDGMENTS

We gratefully acknowledge the support of the Bren School of Environmental Science & Management at the University of California, Santa Barbara. This paper is the product of an interdisciplinary PhD seminar. The first 15 authors were participants. Anderson and Tague were the faculty leads.

REFERENCES

Allen CR, Gunderson LH. 2011. Pathology and failure in the design and implementation of adaptive management. Journal of environmental management 92: 1379-1384.

Au J, Bagchi P, Chen B, Martinez R, Dudley SA, Sorger GJ. 2000. Methodology for public monitoring of total coliforms, Escherichia coli and toxicity in waterways by Canadian high school students. Journal of environmental management 58: 213-230.

Becker CD, Agreda A, Astudillo E, Costantino M, Torres P. 2005. Community-based monitoring of fog capture and biodiversity at Loma Alta, Ecuador enhance social capital and institutional cooperation. Biodiversity and Conservation 14: 2695-2707.

Bois ST, Silander JA, Mehrhoff LJ. 2011. Invasive Plant Atlas of New England: The Role of Citizens in the Science of Invasive Alien Species Detection. Bioscience 61: 763-770.

Bone J, Archer M, Barraclough D, Eggleton P, Flight D, Head M, Jones DT, Scheib C, Voulvoulis N. 2012. Public Participation in Soil Surveys: Lessons from a Pilot Study in England. Environmental Science & Technology 46: 3687-3696.

Bonney R. 2007. Citizen science at the Cornell Lab of Ornithology. Exemplary science in informal education settings: Standards-based success stories: 213-229.

Bonney R, Ballard H, Jordan R, McCallie E, Phillips T, Shirk J, Wilderman CC. 2009a. Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for Informal Science Education. A CAISE Inquiry Group Report. Online Submission.

Bonney R, Cooper CB, Dickinson J, Kelling S, Phillips T, Rosenberg KV, Shirk J. 2009b. Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. BioScience 59: 977-984.

Brook RK, Kutz SJ, Veitch AM, Popko RA, Elkin BT, Guthrie G. 2009. Fostering Community-Based Wildlife Health Monitoring and Research in the Canadian North. Ecohealth 6: 266-278.

Brunsdon C, Comber L. 2012. Assessing the changing flowering date of the common lilac in North America: a random coefficient model approach. Geoinformatica 16: 675-690.

Cohn JP. 2008. Citizen science: Can volunteers do real research? BioScience 58: 192-197.

Cooper CB, Dickinson J, Phillips T, Bonney R. 2007. Citizen science as a tool for conservation in residential ecosystems. Ecology and Society 12.

Cornwell ML, Campbell LM. 2012. Co-producing conservation and knowledge: Citizen-based sea turtle monitoring in North Carolina, USA. Social Studies of Science 42: 101-120.

Cox TE, Philippoff J, Baumgartner E, Smith CM. 2012. Expert variability provides perspective on the strengths and weaknesses of citizen-driven intertidal monitoring program. Ecological Applications 22: 1201-1212.

Crall AW, Newman GJ, Jarnevich CS, Stohlgren TJ, Waller DM, Graham J. 2010. Improving and integrating data on invasive species collected by citizen scientists. Biological Invasions 12: 3419-3428.

Crall AW, Newman GJ, Stohlgren TJ, Holfelder KA, Graham J, Waller DM. 2011. Assessing citizen science data quality: an invasive species case study. Conservation Letters 4: 433-442.

Crewe TL, Timmermans ST, Jones KE. 2005. The Marsh Monitoring Program Annual Report, 1995-2003. Bird Studies Canada, Port Rowan, Ontario.

Dangles O, Carpio FC, Villares M, Yumisaca F, Liger B, Rebaudo F, Silvain JF. 2010. Community-Based Participatory Research Helps Farmers and Scientists to Manage Invasive Pests in the Ecuadorian Andes. Ambio 39: 325-335.

Danielsen F, Burgess ND, Balmford A. 2005. Monitoring matters: examining the potential of locally-based approaches. Biodiversity & Conservation 14: 2507-2542.

Davies AL, White RM. 2012. Collaboration in natural resource governance: Reconciling stakeholder expectations in deer management in Scotland. Journal of environmental management 112: 160-169.

Devictor V, Whittaker RJ, Beltrame C. 2010. Beyond scarcity: citizen science programmes as useful tools for conservation biogeography. Diversity and Distributions 16: 354-362.

Dickinson JL, Zuckerberg B, Bonter DN. 2010. Citizen Science as an Ecological Research Tool: Challenges and Benefits. Annual Review of Ecology, Evolution, and Systematics, Vol 41 41: 149-172.

Doremus H. 2010. Adaptive Management as an Information problem. NCL Rev. 89: 1455.

Downs CT. 2005. Abundance of the endangered Cape parrot, Poicephalus robustus, in South Africa: implications for its survival. African Zoology 40: 15-24.

Ellul C, Francis L, Haklay M. 2011. A Flexible Database-Centric Platform for Citizen Science Data Capture. e-Science Workshops (eScienceW), 2011 IEEE Seventh International Conference on, p39-44.

Farmer RG, Leonard ML, Horn AG. 2012. Observer Effects and Avian-Call-Count Survey Quality: Rare-Species Biases and Overconfidence. Auk 129: 76-86.

Fernandez-Gimenez ME, Ballard HL, Sturtevant VE. 2008. Adaptive Management and Social Learning in Collaborative and Community-Based Monitoring: a Study of Five Community-Based Forestry Organizations in the western USA. Ecology and Society 13.

Finn PG, Udy NS, Baltais SJ, Price K, Coles L. 2010. Assessing the quality of seagrass data collected by community volunteers in Moreton Bay Marine Park, Australia. Environmental Conservation 37: 83-89.

Fitzpatrick MC, Preisser EL, Ellison AM, Elkinton JS. 2009. Observer bias and the detection of low-density populations. Ecological Applications 19: 1673-1679.

Fleiss JL. 1971. Measuring Nominal Scale Agreement among Many Raters. Psychological Bulletin 76: 378-382.

Fleming JR, Love EA. 2012. Healthy Lakes and Vibrant Economies: Linking History, Sense of Place, and Watershed Protection in the Belgrade Lakes Region. Maine Policy Review 21: 90-94.

Foster-Smith J, Evans SM. 2003. The value of marine ecological data collected by volunteers. Biological Conservation 113: 199-213.

Gardiner MM, Allee LL, Brown PMJ, Losey JE, Roy HE, Smyth RR. 2012. Lessons from lady beetles: accuracy of monitoring data from US and UK citizen-science programs. Frontiers in Ecology and the Environment 10: 471-476.

Gregory R, Ohlson D, Arvai J. 2006. Deconstructing adaptive management: criteria for applications to environmental management. Ecological Applications 16: 2411-2425.

Gunderson L. 1999. Resilience, flexibility and adaptive management—antidotes for spurious certitude. Conservation ecology 3: 7.

Gura T. 2013. Citizen science: amateur experts. Nature 496: 259-261.

Habron G. 2003. Role of adaptive management for watershed councils. Environmental Management 31: 29-41.

Hamilton RJ, Giningele M, Aswani S, Ecochard JL. 2012. Fishing in the dark-local knowledge, night spearfishing and spawning aggregations in the Western Solomon Islands. Biological Conservation 145: 246-257.

Holck MH. 2008. Participatory forest monitoring: an assessment of the accuracy of simple cost–effective methods. Biodiversity and Conservation 17: 2023-2036.

Holling CS. 1978. Adaptive environmental assessment and management. Adaptive environmental assessment and management.

Hoyer MV, Wellendorf N, Frydenborg R, Bartlett D, Canfield DE. 2012. A comparison between professionally (Florida Department of Environmental Protection) and volunteer (Florida LAKEWATCH) collected trophic state chemistry data in Florida. Lake and Reservoir Management 28: 277-281.

Humber F, Godley BJ, Ramahery V, Broderick AC. 2011. Using community members to assess artisanal fisheries: the marine turtle fishery in Madagascar. Animal Conservation 14: 175-185.

Jetz W, McPherson JM, Guralnick RP. 2012. Integrating biodiversity distribution knowledge: toward a global map of life. Trends in Ecology & Evolution 27: 151-159.

Johnson BL. 1999. The role of adaptive management as an operational approach for resource management agencies.

Jordan RC, Gray SA, Howe DV, Brooks WR, Ehrenfeld JG. 2011. Knowledge Gain and Behavioral Change in Citizen-Science Programs

Obtención de Conocimiento y Cambio Conductual en los Programas de Ciencia-Ciudadana. Conservation Biology 25: 1148-1154.

Keith DA, Martin TG, McDonald-Madden E, Walters C. 2011. Uncertainty and adaptive management for biodiversity conservation. Biological Conservation 144: 1175-1178.

Landis JR, Koch GG. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics 33: 159-174.

Lee KN. 1999. Appraising adaptive management. Conservation ecology 3: 3.

Leopold M, Cakacaka A, Meo S, Sikolia J, Lecchini D. 2009. Evaluation of the effectiveness of three underwater reef fish monitoring methods in Fiji. Biodiversity and Conservation 18: 3367-3382.

Levrel H, Fontaine B, Henry PY, Jiguet F, Julliard R, Kerbiriou C, Couvet D. 2010. Balancing state and volunteer investment in biodiversity monitoring for the implementation of CBD indicators: A French example. Ecological Economics 69: 1580-1586.

Linkov I, Satterstrom FK, Kiker G, Batchelor C, Bridges T, Ferguson E. 2006. From comparative risk assessment to multi-criteria decision analysis and adaptive management: Recent developments and applications. Environment International 32: 1072-1093.

Lombard M, Snyder‐Duch J, Bracken CC. 2002. Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human communication research 28: 587-604.

Lotz A, Allen CR. 2007. Observer bias in anuran call surveys. Journal of Wildlife Management 71: 675-679.

Lye GC, Osborne JL, Park KJ, Goulson D. 2012. Using citizen science to monitor Bombus populations in the UK: nesting ecology and relative abundance in the urban environment. Journal of Insect Conservation 16: 697-707.

Lyons JE, Runge MC, Laskowski HP, Kendall WL. 2008. Monitoring in the Context of Structured Decision-Making and Adaptive Management. Journal of Wildlife Management 72: 1683-1692.

McKenzie L, Campbell S, Roder C. 2001. Seagrass-Watch: Manual for mapping and monitoring seagrass resources by community (citizen) volunteers. QFS, NFC, Cairns) 94pp.

McKenzie L, Long L, Coles R, Roder C. 2000. Seagrass-Watch: Community based monitoring of seagrass resources. Biologia marina mediterranea 7: 393-396.

Meffe G, Nielsen L, Knight RL, Schenborn D. 2010. Ecosystem management: adaptive, community-based conservation: Island press.

Metzger ES, Lendvay JM. 2006. Seeking Environmental Justice through Public Participation: A Community-Based Water Quality Assessment in Bayview Hunters Point. Environmental Practice 8: 104-114.

Miller IR, De'ath G. 1996. Effects of training on observer performance in assessing benthic cover by means of the manta tow technique. Marine and freshwater research 47: 19-26.

Monroe MC, Plate R, Oxarart A. 2013. Intermediate Collaborative Adaptive Management Strategies Build Stakeholder Capacity. Ecology and Society 18.

Munson MA, Caruana R, Fink D, Hochachka WM, Iliff M, Rosenberg KV, Sheldon D, Sullivan BL, Wood C, Kelling S. 2010. A method for measuring the relative information content of data from different monitoring protocols. Methods in Ecology and Evolution 1: 263-273.

Nichols JD, Runge MC, Johnson FA, Williams BK. 2007. Adaptive harvest management of North American waterfowl populations: a brief history and future prospects. Journal of Ornithology 148: S343-S349.

Nichols JD, Williams BK. 2006. Monitoring for conservation. Trends in Ecology & Evolution 21: 668-673.

Oldekop JA, Bebbington AJ, Berdel F, Truelove NK, Wiersberg T, Preziosi RF. 2011. Testing the accuracy of non-experts in biodiversity monitoring exercises using fern species richness in the Ecuadorian Amazon. Biodiversity and Conservation 20: 2615-2626.

Pattengill-Semmens CV, Semmens BX. 2003. Conservation and management applications of the reef volunteer fish monitoring program. Environmental Monitoring and Assessment 81: 43-50.

Pittman SE, Dorcas ME. 2006. Catawba River corridor coverboard program: a citizen science approach to amphibian and reptile inventory. J. NC Acad. Sci 122: 142-151.

Plummer R, Armitage DR. 2007. Charting the new territory of adaptive co-management: A Delphi study. Ecology and Society 12.

Prell C, Hubacek K, Reed M. 2009. Stakeholder Analysis and Social Network Analysis in Natural Resource Management. Society & Natural Resources 22: 501-518.

Reed MS. 2008. Stakeholder participation for environmental management: A literature review. Biological Conservation 141: 2417-2431.

Reed MS, Graves A, Dandy N, Posthumus H, Hubacek K, Morris J, Prell C, Quinn CH, Stringer LC. 2009. Who's in and why? A typology of stakeholder analysis methods for natural resource management. Journal of Environmental Management 90: 1933-1949.

Reever Morghan KJ, Sheley RL, Svejcar TJ. 2006. Successful adaptive management-the integration of research and management. Rangeland Ecology & Management 59: 216-219.

Ringold PL, Alegria J, Czaplewski RL, Mulder BS, Tolle T, Burnett K. 1996. Adaptive monitoring design for ecosystem management. Ecological Applications 6: 745-747.

Rist L, Campbell BM, Frost P. 2013. Adaptive management: where are we now? Environmental Conservation 40: 5-18.

Sale PF, Cowen RK, Danilowicz BS, Jones GP, Kritzer JP, Lindeman KC, Planes S, Polunin NVC, Russ GR, Sadovy YJ, Steneck RS. 2005. Critical science gaps impede use of no-take fishery reserves. Trends in Ecology & Evolution 20: 74-80.

Sharpe A, Conrad C. 2006. Community based ecological monitoring in Nova Scotia: Challenges and opportunities. Environmental Monitoring and Assessment 113: 395-409.

Shea CP, Peterson JT, Wisniewski JM, Johnson NA. 2011. Misidentification of freshwater mussel species (Bivalvia: Unionidae): contributing factors, management implications, and potential solutions. Journal of the North American Benthological Society 30: 446-458.

Shirk JL, Ballard HL, Wilderman CC, Phillips T, Wiggins A, Jordan R, McCallie E, Minarchek M, Lewenstein BV, Krasny ME, Bonney R. 2012. Public Participation in Scientific Research: a Framework for Deliberate Design. Ecology and Society 17.

Silvertown J. 2009. A new dawn for citizen science. Trends in Ecology & Evolution 24: 467-471.

Silvertown J, Buesching CD, Jacobson SK, Rebelo T. 2013. Citizen science and nature conservation. Key Topics in Conservation Biology 2: 127-142.

Sunde P, Jessen L. 2013. It counts who counts: an experimental evaluation of the importance of observer effects on spotlight count estimates. European Journal of Wildlife Research 59: 645-653.

Susskind L, Camacho AE, Schenk T. 2012. A critical assessment of collaborative adaptive management in practice. Journal of Applied Ecology 49: 47-51.

Szabo JK, Vesk PA, Baxter PWJ, Possingham HP. 2010. Regional avian species declines estimated from volunteer-collected long-term data using List Length Analysis. Ecological Applications 20: 2157-2169.

Turnhout E, Van Bommel S, Aarts N. 2010. How participation creates citizens: participatory governance as performative practice. Ecology and Society 15(4): 26.

Van Rijsoort J, Zhang JF. 2005. Participatory resource monitoring as a means for promoting social change in Yunnan, China. Biodiversity and Conservation 14: 2543-2573.

Walters C. 1997. Challenges in adaptive management of riparian and coastal ecosystems. Conservation ecology 1: 1.

Walters CJ. 2007. Is adaptive management helping to solve fisheries problems? AMBIO: A Journal of the Human Environment 36: 304-307.

Walters CJ, Hilborn R. 1976. Adaptive control of fishing systems. Journal of the Fisheries Board of Canada 33: 145-159.

Westgate MJ, Likens GE, Lindenmayer DB. 2013. Adaptive management of biological systems: A review. Biological Conservation 158: 128-139.

Williams BK. 2011. Adaptive management of natural resources—framework and issues. Journal of Environmental Management 92: 1346-1353.

Yaffee SL. 2002. Ecosystem Management in Policy and Practice. Meffe GK, Nielsen LA, Knight RL, Schenborn DA editors. Ecosystem Management: Adaptive Community-Based Conservation. Washington, D. C.: Island Press, p89-93.

FIGURE LEGEND

Figure 1

STAKE: a five-point framework of the kinds of incentives that motivate citizen scientists to become engaged stakeholders in natural resource management

TABLE LEGEND

Table 1 Monitoring and Stakeholder Engagement Criteria for Successful Adaptive Management
Monitoring	Stakeholder Engagement
MC1	Monitoring must take place	SC1	Stakeholders must be identified and engaged in the management process.
MC2	Data must be relevant to management actions, quantitative, and subject to quality and consistency controls.	SC2	Managers must provide appropriate incentives for participation.
MC3	Monitoring must be cost-effective	SC3	Decision-makers must be accountable to stakeholders.
MC4	Monitoring must occur at appropriate temporal and spatial scales

Conceived of and designed study (EAB, ASA, DB, WB, PC, MF, KG, RG, YH, IM, IP, SS, WW, YY, TZ, SA, CT), performed research (EAB, ASA, DB, WB, PC, MF, KG, RG, YH, IM, IP, SS, WW, YY, TZ), analyzed data (EAB, ASA, DB, WB, PC, MF, KG, RG, YH, IM, IP, SS, WW, YY, TZ, SA, CT), wrote paper (EAB, ASA, DB, WB, PC, MF, KG, RG, YH, IM, IP, SS, WW, YY, TZ, SA, CT)

Of course, there may be other reasons for the failure of citizen science to contribute to management. For example, managers might be reluctant to use citizen science-collected data. We focus here on the two most commonly-cited reasons for failure, while acknowledging that the use of citizen science for adaptive management may face other barriers.

Other scholars (Bonnery, et al., 2009a) have distinguished between contributory, collaborative and co-created models for PPSR in order to assess whether citizen science can contribute to informal science education. While such a categorization could also be useful for determining when citizen science effectively engages stakeholders, we opt to distinguish instead between types of participants, in part because extracting the information about informal learning was usually not possible from the studies reviewed.

We caution that the search terms we used likely do not cover all of the studies that involve citizen scientists, since terminology is highly variable. Although some programs date back to the late 1800s, we choose to limit the study to more recent papers, following Bonney et al. (2009) who argue that “The current concept of citizen science, however, with its integration of explicit and tested protocols for collecting data, vetting of data by professional biologists, and inclusion of specific and measurable goals for public education, has evolved primarily over the past two decades (Bonney 2007, Cohn 2008).” Including earlier papers that are more likely to lack a rigorous protocol would bias our results toward finding that citizen science cannot be useful for adaptive management. Although we inevitably miss some studies from our time period, there is no reason to think that the bias introduced by these omissions makes us more or less likely to conclude that citizen science can be used to resolve the two main problems of adaptive management.