We academics tend to spoil many a good energy access party with pesky questions about “impact”. While the advocacy world often welcomes new innovations and ideas with a warm embrace and enthusiasm, academics tend to be skeptical and call for lengthy randomized controlled trials and further research. Despite shared goals, academics and advocates just don’t see eye to eye on impact evaluation.

One reason why impact evaluation remains contested is that the very definition of impact is ambiguous. Establishing that life without modern energy access is hard and not very productive really does not require further research. Any intelligent human being can see that life without electricity, clean cooking fuels, and motorized transportation is inherently limited.

This basic observation provokes different reactions. Advocates react by underscoring the urgency of action and expressing frustration with yet another call for further research. For this tribe, the problem is clear and the primary issue a lack of action.

Members of the academic tribe react quite differently. We note that a shared understanding of the flawed status quo does not mean we know what we should do. For us, the problem is also clear, but whether and how we should act on it remains far from clear.

The academic argument against rapid action has two prongs. The first is that we don’t yet have a very good understanding of the impacts of different interventions. Even if we all agree that cooking on traditional biomass is dangerous and unproductive, that doesn’t tell us whether any particular LPG subsidy or cookstove distribution program would improve lives.

The second is that even if an energy access intervention is beneficial, there could be even better interventions available. Instead of an LPG subsidy, we could just give people money and let them freely choose their own preferred goods and services. This is perhaps a shocking idea, but that’s how most of us living in great abundance are paid – my salary is paid in $$$ not kilowatt-hours or kerosene.

We all agree that the lack of modern energy access is a human tragedy. We all agree that modern life requires a massive expansion of access to modern energy services in developing countries. In that sense, the advocates have it exactly right – we can no longer ignore this problem.

But that doesn’t mean we really understand what to do. Impact evaluation is at times an awkward exercise, because many of the brilliant from Paris or San Francisco don’t actually work in the field. If we are serious about improving the lives of the energy-poor, we must subject any and all new ideas to impact evaluation as soon as they are ready for prime time.