A Crystal Ball for Forests: Using Today's Indicators to Predict Tomorrow's Impacts

Worldwide, an estimated 1.3 billion people - the majority of whom subsist on less than $1.25 per day - depend directly on forests for their livelihoods.[i] Therefore, it would seem clear that programs to restore and maintain forests must also contribute to poverty alleviation. But is this true, and if so, how do we know?

On one hand, there is increasing demand for better evidence on program effectiveness, including in the forestry sector. On the other hand, many programs – particularly those focused on natural resource-based development – can only be assessed properly by looking at long-term outcomes. As a result, there is a mismatch between the urgency with which global poverty and deforestation issues need to be addressed, and the time that it takes to get results.

To help close this knowledge gap, a World Bank team received PROFOR funding to develop a set of Predictive Proxy Indicators (PPIs) – measures of progress made while a project or policy is still ongoing, but that provide a forecast for a longer-term impact. PPIs are often used in other fields: for example, education level is widely used as a predictor for future earnings. But use of PPIs is a new development for the forestry sector, with the potential not just to measure the impacts of forestry programs on poverty reduction and economic growth, but also on other important development outcomes, such as biodiversity conservation, climate change mitigation and adaptation, and good governance.

“This is a long-standing issue with evaluating forestry projects,” notes lead author and Senior Forestry Specialist Daniel Miller. “It takes years for trees to mature to a point where they deliver environmental benefits, let alone economic ones, but funding cycles are much shorter so program evaluation may not reflect the benefits that accrue later on. Predictive proxies are a cost-effective complement to rigorous impact evaluation, with the potential to address the issue of different time lags.”

To maximize the accuracy of their predictive power, each PPI is composed of a cluster of indicators that, taken together, was determined to have strong predictive potential. In addition, each PPI is based on a plausible theory of change that explains why the PPI was likely to predict a certain outcome as a result of an intervention. For instance, the PPI for Sustainable, Forest-Related Income, which has poverty reduction as an objective, is made up of three indicators, as seen in the following figure:

One of the report’s main findings was that PPIs for forestry do in fact exist and can be used in practice. This, according to Miller, was not at all a foregone conclusion at the start of the project: “Reliable outcome indicators are usually simple to use once they’ve been developed, but identifying them can be difficult. We began this research knowing that we might fail to find anything functional. We’d never seen the PPI approach used in forestry before, so I think it’s an innovative way of solving a fundamental challenge in the sector.”

Encouragingly, the Core Sector Indicators (CSIs) already used by the World Bank have strong potential as PPIs, meaning that CSIs can help to capture not only end-of project outcomes, but also longer-term impacts of forest investments, and in a consistent way across countries and contexts. According to the report, another reason to use detailed monitoring and evaluation (M&E) strategies such as PPIs is that M&E investments are associated with higher project success: Projects with a highly satisfactory M&E component were more likely to have a highly satisfactory outcome rating too, compared to a project with only a satisfactory M&E score.  

PPIs are already being integrated into new World Bank projects, such as the Argentina Forests and Community Project. This initiative aims to increase access to markets and basic services by small forest producers, including indigenous people and campesinos, in Argentina’s comparatively poor but forest-rich Northern provinces. Activities will focus on helping communities to develop Sustainable Forest Management (SFM) plans and to strengthen their tenure. To assess effectiveness, the project team will collect data on the three indicators under the Sustainable Forest-Related Income PPI. The project will also be assessed using rigorous impact evaluation methods, making it the first forward-looking test case of how well the forestry PPIs perform as predictors of future outcomes.

 

For stories and updates on related activities, follow us on twitter and facebook, or subscribe to our mailing list for regular updates.

Last Updated : 12-16-2016