There have been some studies considering the relationship between research and environmental policy but studies capturing the impact of research on environmental management, environmental policy, and environmental outcomes are relatively rare. Here is one attempt.

Environmental research may generate benefits in a variety of ways including by providing: information or technology that allows improved management of an environmental issue; information that fosters improved decision-making about priorities for environmental management or policy; or information about an environmental issue that is of intrinsic interest to the community. There are several reasons why it can be worth measuring the impacts of environmental research, including making a case for the funding of environmental research, informing decisions about research priorities, and helping researchers to make decisions about their research that increase its ultimate benefits.

Earlier this year we released the results of an assessment of the engagement and impacts of a particular environmental research centre, the ARC Centre of Excellence for Environmental Decisions (CEED). The assessment includes impacts on policy, management and the community, as well as measures of academic performance, including publications, citations and collaborations. Data were collected in several ways: a survey of all project leaders for the Centre’s 87 projects, the preparation of detailed case studies for selected projects, and collection of statistics on publications, citations and collaborations.

The approach taken was informed by a recent paper of ours called “Policy-oriented environmental research: What is it worth?” (Pannell et al. 2018). The full report is available here.

The Centre’s engagement with end users and stakeholders was strong in Australia and around the world. Researchers reported many examples of engagement with research users involved in policy and management. Results were highly heterogeneous and somewhat skewed, with the majority of observed impact occurring in a minority of the projects.

For almost half of the projects, the potential future increase in impact was assessed as being moderate or high. To some extent, this reflects the time lags involved in research attempting to influence policy and management, but the information was also used to identify projects for which additional engagement effort could be beneficial. The correlation between impact and academic performance was positive but low.

To obtain richer detail about impacts, detailed case studies were prepared for nine research projects. The projects were selected to be diverse, rather than representative. These case studies highlight the unique circumstances faced by each project in endeavouring to have an impact. Each project must be framed within a strong understanding its domain and be deeply engaged with research users if impact is to occur. Substantial benefits for policy or management are apparent in a number of the case studies.

A factor contributing greatly to the impact of CEED was the research communication magazine Decision Point. This publication was widely accepted as a valued communication resource for academic findings in the field of environmental decision sciences, and was rated by people in government and academic institutions as relevant and informative.

Some valuable lessons and implications of the impact analysis are identified in the report. Research impact does not depend only on good relationships, engagement and communication, but also importantly on what research is done. Therefore, embedding a research culture that values impact and considers how it may be achieved before the selection of research projects is potentially important. The role of the Centre leadership team in this is critical. Embedding impact into the culture of a centre likely occurs more effectively if expertise in project evaluation is available internally, either through training or appointments.

A challenge in conducting this analysis was obtaining information related to engagement and impact. There may be merit in institutionalising the collection of impact-related data from early in the life of a new research centre.

Interestingly, we found little relationship between (a) impact from translation and engagement and (b) measures of academic merit. It should not be presumed that the most impactful projects will be those of greatest academic performance.

At the time of the assessment, CEED had generated 848 publications which had been cited 14,996 times according to the Web of Science. CEED publications are disproportionately among the most cited papers in their disciplines. More than a quarter of CEED publications are in the top 10% of the literature, based on their citations. For 39 CEED publications (one in 22), their citations place them in the top 1% of their academic fields in the past 10 years.

There are often long lags between the start of research and delivering the impact — decades in many cases. Therefore, there is a need to allow the longest possible time lag when assessing research impact. On shorter timescales, it may be possible to detect engagement, but not the full impact that will eventually result.

Further reading

Pannell, D.J., Alston, J.M., Jeffrey, S., Buckley, Y.M., Vesk, P., Rhode, J.R., McDonald-Madden, E., Nally, S., Gouche, G. and Thamo, T. (2018). Policy-oriented environmental research: What is it worth? Environmental Science and Policy 86, 64-71. Journal web page

Thamo, T., Harold, T., Polyakov, M. and Pannell, D. (2018). Assessment of Engagement and Impact for the ARC Centre of Excellence for Environmental Decisions, CEED, University of Queensland. http://ceed.edu.au/resources/impact-report.html