I struck a particularly prickly questioner after a conference presentation I gave in Sydney two weeks ago. The questioner clearly had a very jaundiced view of my work, and, I suspect, of economics in general. Responding to such a person is challenging, but can actually be quite helpful sometimes, as it brings issues to the surface that otherwise stay buried. I use it here as an opportunity to outline an approach we’ve developed to make economics relevant, accessible and useful in supporting decision makers.

When you give a conference presentation and, in question time, someone starts their question with, “With respect, …”, you know you’re in trouble. “With respect, …” generally means “I have a deeply held and fundamental disagreement with what you’ve just said”, and that was exactly the case in this instance.

The context was a national conference in Sydney organised by the Bushfire Cooperative Research Centre and the Australasian Fire and Emergency Service Authorities Council. My talk was about a project we are just starting, doing integrated assessment of various strategies for prescribed burning to reduce risks to life and property.

My questioner seemed to have (at least) three big problems with my presentation.

1. She objected to the idea that we could express important values and ideas in numbers. Or perhaps it was something like, “you can’t express everything that matters in numbers”.
2. Probably related to point 1, she suspected that we would exclude important issues from the analysis.
3. She seemed to feel that we would be taking control of the agenda and the decision process, imposing obnoxious economist values and economist judgments. To be fair, she didn’t actually say “obnoxious”, but it seemed to be the subtext.

The approach we’ll be using in the project is based on 10 years experience of trying to help policy makers and managers make better decisions about complex environmental problems. The aim is not to turn people into economists, but to help them make decisions that are more in line with the goals of the programs they are running or delivering. The approach is to ask, how can we achieve the goal or goals to the highest possible level with the available resource?. In the case of bushfires, there are multiple goals that may need to be traded off (e.g. protecting life, property, water resources, the environment). We will be teasing out the highest combinations of these outcomes that can be achieved, and quantifying the trade-offs between them.

The approach involves working closely with policy makers and managers to understand the decision options that they think are potentially feasible, and the values associated with the outcomes that will be delivered. We use the available research and judgments from researchers and local stakeholders to identify what those outcomes would be, and to estimate the likely behavioural responses of people and organisations affected by the project. We bring all that together in a quantitative tool that allows us (and any stakeholder) to ask what-if questions about different decision options.

Having clarified that, will the project line up with our questioner’s low expectations?

Starting with number 3, we certainly will not be taking control of the agenda or the decision process. The aim is to expand and clarify the options that are available, and to assess their consequences. The broad goals against which the options will be assessed are those of the funders and decision makers, not ours. Our job, in part, is to be advocates for the public interest, assessing whether particular strategies or interventions will contribute outcomes sufficient to justify the resources they use up, given that those resources could be used in other projects to pursue the same goals, or different goals.

We appreciate that the decisions are not up to us. We provide information, and others make the decisions. They are likely to include considerations beyond our analysis (e.g. political consequences). For that reason, we tend not to say which options are the absolute best, although we do indicate that some options are clearly not worth doing, usually because they are not technically or socially feasible given the budget they are likely to have available.

It’s true that judgments will be required at times, but even there we use other experts to make them if possible. For example, there is often a lack of clear evidence to support a particular number to feed into the analysis. We collect the available evidence and ask relevant experts to help us by selecting a best-bet value, preferably by consensus. If consensus is not possible (e.g. there are differing scientific opinions on the matter), we include different values, and tease out their consequences, to see if the disagreement actually makes any important difference. Sometimes it does, sometimes it doesn’t.

Sometimes, on particular issues, the best expert might be one of our team — that’s fine. For example, that might be the case in relation to farmer adoption of new practices, or the farm-level economics of a land management option.

Although the approach is grounded in economics, we are not just economists. We also have expertise in biological, physical and social sciences, and we collaborate closely with experts from all disciplines. In 2009, we won the Eureka Prize for Excellence in Interdisciplinary Research.

Back to point 2 of the questioner’s list of concerns. Obviously it’s true that an analysis might not explicitly factor in every conceivable relevant consideration. However, if the decision makers and stakeholders we work with think something is important and ask us to factor it in, we’ll try to do so. If something is excluded, it is because (a) the the decision makers and stakeholders haven’t thought it important enough to include, or (b) there is too little information or understanding about the issue to warrant its inclusion.

The exception there is politics, which we don’t include. We’re interested in promoting real outcomes for the community, not political outcomes. If a decision maker promotes an option for political reasons, we want it to be clear what the resulting cost is in terms of lost community outcomes.

Finally, there is our questioner’s aversion to numbers. Where appropriate, we do actually capture an issue qualitatively, rather than quantitatively, and report something to the decision makers in words, rather than numbers. One of a number of possible examples here is the quality of the information used in the analysis. It’s important for decision makers to know whether the analysis of a decision option is based on really strong evidence, or expert judgments based on minimal evidence. We capture and report that qualitatively, not in numbers.

But where it is possible, and makes sense, doing a numerical analysis of the pros and cons of each decision option is crucial. Our experience provides abundant examples that reinforce this. It is not sufficient to describe all of the benefits and costs qualitatively, because every decision option has both benefits and costs. For example, a purely qualitative approach can’t tell you that, for option 3, the ratio of benefits to costs is 10 times greater than for option 7, which is essential information for sound decision making.

I should acknowledge that our approach to this sort of research is actually relatively unusual for economists. Experience over a decade has helped us to develop an approach that makes economics accessible and helpful to designers, managers and deliverers of programs. It is pragmatic and participatory, but still firmly based on sound economic theories and principles. We make sure that the biological, physical and social science we use is not compromised to suit over-simplified economic models, which, sadly, happens too often in the mainstream economics literature.

This is not to say ours is the only relevant approach to economic analysis, even for these sorts of problems. It addresses a sub-set of the possible economic questions that are relevant to a particular group of decision makers. By demonstrating the value and usefulness of economic thinking and analysis, it may actually provide a vehicle for introducing more complex economic models to these decision makers.

David Pannell, The University of Western Australia

Further reading

Pannell, D.J. and Roberts, A.M. (2009). Conducting and delivering integrated research to influence land-use policy: salinity policy in Australia, Environmental Science and Policy 12(8): 1088-1099. Full paper (94K) – Journal version on-line

Pannell, D.J. (2009). Why don’t environmental managers use decision theory? Pannell Discussions, no. 150, http://dpannell.fnas.uwa.edu.au/pd/pd0150.htm.