In a newsletter editorial last week, Hugh Possingham of University of Queensland asks the following question. Given the strong potential benefits to the environment from using formal decision tools (consistent with decision theory) to prioritise environmental investments, why don’t environmental managers and policy agencies use such tools more? He discusses a number of reasons. I’ve got some more.

I’ve invested a lot of time and effort in the past five years developing decision tools for environmental managers, and trying to get them actually used. We’ve made some progress. Currently about 14 of Australia’s 56 regional Natural Resource Management bodies have at least trialed INFFER, and it is clearly influencing the way some people think about the issues. We’ve also made some mistakes and learnt some lessons the hard way. Given all this, I’ve got a strong perspective on Hugh’s question.

Hugh discusses three possible contributing reasons (see here), the first two provided to him by Max Bourke.

1. “It [decision theory] is ‘too rational’ and does not take account of the socio-political environment which of course clouds all decision making.” I agree with Max and Hugh that this is probably a factor, but it shouldn’t be. Decision tools don’t necessarily need to take account of the socio-political environment to be useful. People do that. The tools just have to provide useful information. (Having said that, we do take account of some aspects of the socio-political environment in INFFER, because given what INFFER aims to do, it’s important.)
2. “It has potential in a perfect market, but these don’t exist.” This seems to imply that decision theory tools only have potential in a perfect market. If that’s the message, I don’t agree at all. It’s precisely because markets are not perfect that you need such tools. Maybe Max means something different than my interpretation.
3. “Decision theory diminishes the influence of decision-makers, and in general they don’t like that.” Certainly this is sometime true. In talking about INFFER to a ministerial advisor, she was concerned about whether it would reduce the Minister’s discretion.

There are many other reasons as well, and different ones come into play at different time. I group the reasons for non-adoption into three categories: those related to the tools and their capabilities (point 1 is one of those), those related to the intended users and their capacities, and those related to the institutions that the tools would have to work within (point 3 is one of those). If you want to get your decision tool actually used, you’ve got to pay attention to all three aspects: the tool itself, the users, and the institutional environment.

Here are some other possible reasons for non-adoption of decision theory tools that may be relevant in particular cases.

The tools

The tools require a level of expertise or knowledge that users do not possess. For example, they may be comfortable dealing with technical information, but uncomfortable with socio-economic information.

The tools are not sufficiently flexible for different people to use them in ways that suit them. The literature on adoption of innovation shows that users often adapt innovations to suit their own circumstances.

The tools are too complex, making them difficult and expensive to use. In INFFER we’ve put a lot of effort into making the tools as simple as possible (e.g. Pannell, 2008; Roberts and Pannell, 2009) but even then many people find them challenging.

The tools themselves are not up to the job, for various reasons. They answer the wrong question, or require data that isn’t available.

The tools contain bugs. It doesn’t take much of a bug for potential users to lose confidence.

The tools are poorly documented and lack adequate help facilities.

Most of the above problems can probably be summarised under one point: The tools were developed without adequate consultation and collaboration with intended users.

The users

Potential users are not aware of the available tools. Developers of decision tools are not good at explaining them, or may have made no attempt to promote them to the relevant people. Users are very busy with other responsibilities and don’t have time to search for or evaluate decision tools.

There are too many tools available, and it’s hard for users to tell what they are good for and which ones are worth using. Scientists don’t provide sufficiently clear and helpful advice about this when asked. (I’ve seen an example of that recently.)

Users have had bad experiences with decision tools or scientists in the past.

Intended users are suspicious of the tools, or suspicious of any scientist who is perceived to be promoting his or her own work. Users don’t distinguish adequately between (a) scientists who are attempting to advance the public interest and (b) the general run of snake-oil salesmen and vested interests who they have to deal with all the time.

It is believed (quite possibly correctly) that the tool’s developers do not understand the policy environment or the relevant decision problems adequately.

Users have a particular world view that conflicts with some aspect of the tool. For example, I’ve had people object to the very idea of prioritising environmental investments because they believe that we should be encouraging and supporting all landholders to manage sustainably, or because it smacks of defeatism to admit that we can’t save everything. The very idea of taking the necessary time to collect and integrate all of the required information to make a more balanced decision is quite a novelty to some users we have worked with. (It seems such a fuss! Is it really necessary?) In other words, it may not be just a matter of adopting a new tool, but rather requires a substantial change of mind set.

The intended users lack the skills or knowledge needed to use the tools (e.g. Pannell et al. 2008). They have not had the training that would be required.

The institutions

There are three essential aspects of the institutional arrangements that will affect the success of a decision tool: (a) training and support, (b) the incentives that exist for using the tool, and (c) the processes into which the tool’s output would have to fit.

(a) Training and support

There is a need for training materials and one-on-one support, but these are not available.

(b) Incentives for use

There are inadequate incentives for people to use the tools. People need a strong reason to change what they are doing. Some people may be prompted to action by the opportunity to do a better job, but in general one should expect that there will need to be carrots (rewards for use) or sticks (penalties for non-use). For example, in getting INFFER used at the regional level, we believe it is essential for governments to give clear signals to the regions that there will be benefits to regions who do use rigorous tools to develop and assess projects. It should be clear that they stand a better chance of getting funds for their proposed projects if the projects have been evaluated positively using an appropriate assessment tool. At the level of a policy agency, the incentive to head down this line might be pressure from a minister for better evaluated projects, or from a body like the Australian National Audit Office.

These carrots and sticks probably need to be strong enough to offset disincentives for use of decision tools that are built into the system. Disincentives can include that:

• There is no perceived need for improved decision tools. There is already a decision making system in place which people feel is adequate. Decision theory tools are more expensive to apply than the existing decision methods. There is a lack of appreciation about how poor existing decisions are.
• Decision theory tools take time to apply, but politics requires quick action. It is administratively or politically difficult to spend money on analyses for decisions that won’t be made until a later year.
• Some people with influence have a vested interest in the existing system. For example, INFFER has faced opposition from a number of people who had a commitment to an existing approach of one sort or another. Some of this opposition has been vociferous and energetic.
• Given the incredible pressures on bureaucrats and politicians, there is not time to become sufficiently informed about decision tools.
• The decision maker would prefer to maintain their discretion to make decision for reasons other than environmental outcomes.

(c) Processes

There is no point in using a decision tool if there is no scope for it to influence the actual decisions. The decision process needs to be set up such that there are people responsible for overseeing the application of the tool, there is adequate quality assurance (such as expert review of the data and assumptions used) to minimise gaming of the system, there is time for the application of the tool to occur, that the body that makes the decisions has an adequate understanding of the tool and how to interpret its outputs, and that the procedures followed make it possible for the decision tool to have an influence. If these conditions do not exist, you should either give up, or work to bring these conditions into being.

When you recognise all these issues, it is not in the least bit surprising that decision theory is not used more often by environmental managers and policy makers. It is more surprising that it is used at all.

David Pannell, The University of Western Australia

Further Reading

Roberts, A. and Pannell, D. (2009) Piloting a systematic framework for public investment in regional natural resource management: dryland salinity in Australia, Land Use Policy (forthcoming). Full paper (270K) · Journal version on-line

Pannell, D.J. and Roberts, A.M. (2009). Conducting and delivering integrated research to influence land-use policy: salinity policy in Australia, Environmental Science and Policy (forthcoming). Full paper (94K) · Journal version on-line

Pannell, D., Ridley, A., Seymour, E., Marsh, S. and Wilkinson, R. (2008).Capacity building in regional NRM, Issues in prioritisation, planning and implementation of environmental works at the regional level, RIRDC Publication No 08/181 RIRDC Project No UWA-92A, RIRDC, Canberra, Full report (630K)

Pannell, D.J. (2008). Public benefits, private benefits, and policy intervention for land-use change for environmental benefits, Land Economics 84(2): 225-240. Here