Think and act for entrepreneurship in Africa

Typically, current assessments of decentralised electrification projects struggle to establish their tangible impact. Achieving this would necessitate expensive and time-intensive efforts, because it would demand extensive data not only from the equipped localities but also from control localities. Nonetheless, substantiating the presence of an impact is crucial to assist public decision-makers, both at the national and international levels, in expanding the adoption of these solutions.

In a series of four articles for “Entreprenante Afrique,” we introduce the Cafés Lumière mini-grids project in Madagascar and compare the outcomes of various affordable evaluation methods applied to these projects. These methods aim to assess the project’s positive effects on development objectives. In this second of four articles, we explore the feasibility of evaluating impacts through the use of remote sensing data, which is generally a low cost approach.

The contribution of remote sensing to impact analysis

Remote sensing uses high-resolution satellite imagery that covers nearly the entire globe, offering near real-time accessibility at a low cost. This technology can effectively capture various measurable terrestrial phenomena relevant for the study of human activities. For the assessment of electrification projects’ impacts, satellite imagery measurements of night-time luminosity can be employed, once sufficient experience has been gained in interpreting this data. It has been consistently demonstrated that an increase in night-time luminosity correlates well with the increase in electrification over time, even at fine levels of granularity (see Berthélemy, 2022, in The Conversation). It has even been argued that rise in night-time luminosity reflects growth in economic activity (Hu and Yao, 2022).

Nonetheless, critics of this new approach, which relies on observations of natural phenomena partially linked to the outcomes of human activities, raise concerns about potential assessment bias. In our context, night-time luminosity specifically measures the light generated at night, primarily from lighting sources, especially public lighting. It does not have a direct connection with total electricity consumption, of which it typically represents a minor portion, and even less so with its impacts on socio-economic development. While public lighting can influence safety, and consequently economic activity, it can only capture a fraction of the consequences of electrification on human activities.

Application: the Cafés Lumière Mini-grids have a Significant Impact from 2021 onwards

Night-time luminosity can serve as a real-time indicator for detecting the effects of electrification. To illustrate this, we have calculated the average annual night-time luminosity for the 6 equipped localities and compared it to that of 6 similar non-equipped localities, which were randomly selected by Electriciens sans frontières to create a control group. The data was analysed taking into consideration the following constraints :

• The different localities were not equipped at the same time ;
• The Cafés Lumière were progressively set up, with priority given to installing the shop before commissioning the mini-grid. Since the shop has little effect on light emission, the effects of Café Lumière can be initially evaluated by examining the influence of the mini-grid ;
• In 2 localities, Ambatonikolahy and Talata Dondona. public lighting operates during specific hours when the satellite passes overhead (between 12pm and 2am in our case). Other localities prefer to have public lighting from 9pm to midnight and from 5am to 6am.

Figure 1: Assessment of night-time brightness (annual averages 2013-2022) in both equipped and non-equipped localities (radiance measured in w/cm2_sr)

Figure 1 shows a similar trend in the average night-time brightness for both the 6 equipped villages and the 6 control villages up to 2020. However, the equipped group exhibit more brightness in 2021 and 2022. This difference is approximately 10% compared to previous data, which, though modest given the initially low level, is statistically significant. Our findings from Figure 1 are corroborated by a more formally rigorous statistical test that utilizes monthly data for each locality, while also controlling for the seasonal effects and fixed effects associated with each treated or untreated locality. The deployment of the mini-grid leads to an increase in night-time luminosity comparable to that depicted in Figure 1, and this increase is statistically significant.

Concerns about potential bias arising from the presence of street lighting are only potentially justified in the case of 2 out of the 6 localities (Ambatonikolahy and Talata Dondona). In principle, these concerns can be examined by considering that street lighting is often installed subsequent to the mini-grid becoming operational. These fears are not borne out.  In fact, when we attempt to account for both the presence of the mini-grid and street lighting simultaneously, the latter shows no significant effect.

This does not imply that public lighting has no effect; rather, it suggests that we are unable to provide demonstrable evidence of its effect. This limitation may be attributed to the statistical power of the test, given the small number of observations with the presence of public lighting.

In the third article, we will employ more precise data, notably on monthly electricity consumption categorized by usage, which will enable us to assess the contribution of public lighting to the increase in night-time luminosity.

Further reading in the same serie of articles on “Measuring the Impact of Decentralised Electrification Projects”: Cafés Lumière in Madagascar (1/4), Characterisation of the Impacts of Decentralised Electrification Projects on Access to Electricity Using Locality Data (3/4), Characterisation of the Impacts of Decentralised Electrification Projects on Access to Electricity Using Household Data (4/4).

The distribution of electricity outside national grids, facilitated through mini-grids, a generic term encompassing various system sizes (pico, micro, mini, or small), stands as the main means of extending access to electricity for rural populations in Africa . In principle this approach should contribute significantly to achieving the MDG7 (Sustainable Development Goal 7: ensuring universal access to reliable, sustainable, and modern energy services at an affordable cost). However, the mini-grids sector faces significant challenges in making substantial progress. National, and at times international, decision-makers encounter difficulties in scaling up this approach due to a lack of compelling evidence regarding its impact, a void that economists and evaluators have so far struggled to fill.

In a series of four articles, we introduce the Cafés Lumière mini-grids project in Madagascar, highlight the limitations of the standard evaluation conducted for accountability purposes, which fails to offer substantive proof of impact, and explore alternative methods for assessing and documenting the socio-economic impacts of the project.

Cafés Lumière in Madagascar: a Shop, a Welcoming place, and a Mini-grid.

The Café Lumière solution, conceived and developed since 2019 by Electriciens sans frontières, provides, in 6 villages in the Vakinankaratra and Itasy regions of Madagascar, a shop with a photovoltaic solar power system, backed up by batteries and a generator.

The Cafés Lumière are unique in their dual role as a mini-grid and a multifunctional energy hub. In other words, the Café Lumière, usually located in the centre of the village, has a shop which offers services such as mobile phone and lamp charging, and supplies electricity to meet other local demands. Thes multi-service platforms deliver energy services and a welcoming space for productive activities which need electricity. The mini-grid supplies electricity from the Café Lumière via local connections to households, businesses, public lighting, and community services. A portion of the electricity consumption for the community services is funded by a contribution collected from sales to other users, households, and businesses.

Four fundamental principles guide the installation of Cafés Lumière :

1. Ensure a minimum access to a sustainable electricity service for all members of a rural community.
2. Enhance the quality of community services, particularly healthcare and education, by establishing long-term minimum access to electricity.
3. Promote the development of private productive activities.
4. Contribute to a political and regulatory framework that empowers local stakeholders to oversee and sustain Café Lumière facilities and services over the long term.

This solution is based on a public-private partnership which involves the operator, Anka, the Agence de Développement de l’Électrification Rurale (ADER), the Structures Collectives de Gestion Mixte (SCGM) at the village level, and the solution provider, Electriciens sans frontières.

With a presence established in each of the villages involved, the service provider operates in close proximity to isolated rural populations. This proximity allows for the sharing of relevant information and immediate actions (operation and maintenance of facilities, sale of services, etc.). Furthermore, a remote monitoring system for energy production has been implemented, enabling the tracking of Café Lumière activities, which plays a pivotal role in generating monitoring and evaluation data for the project by compiling monthly activity reports. These meticulously detailed data facilitate a comprehensive examination of the project’s impact.

The primary funding source for the project came from the French Development Agency through the Sectoral Innovation Facility for Non-Governmental Organisations (FISONG) and its replication under an NGO Initiative Note (NIONG).

In its approach, the Cafés Lumière project is an innovative solution. Based on multiparty funding, and in alignment with the Sustainable Development Goals (SDGs), the project demands consideration for expansion on a broader scale.

Standard Evaluation for Accountability Purposes

Aid agencies use a standard approach to assess the efficient allocation of funds, commissioning an independent firm to produce a report on project completion. This report serves as the final evaluation of the project, as long as it complies with principles of accountability.

This evaluation reports projected results, aligning more with expected rather than observed impacts. In the case of an infrastructure project, the assessment takes place shortly before the planned launch of the equipment, which is often too premature to gauge the expected medium- or long-term effects. While this evaluation fulfils administrative objectives, it may not necessarily enhance our knowledge on how projects contribute to sustainable development goals.

In the context of the Cafés Lumière initiative, the initial analysis conducted on behalf of the French Development Agency concluded that there has been a significant positive impact on the living conditions of the local populations, especially women, and on the enhancement of public services, including education, access to healthcare facilities, and public safety. The findings from focus groups underscore the substantial support provided by this project for fostering economic vitality and the growth of income-generating activities. These results are encouraging and provide an initial indication of the momentum generated by the Cafés Lumière.

However, it is important to note that the evaluation, completed in January 2021, occurred at a time when not all Cafés Lumière projects had reached their full completion. While the multiservice platforms had been launched for all 6 Cafés, only 3 mini-grids were operational, with just 1 having been in operation for over a year. Consequently, the information base collected was too limited in terms of both its timeframe and geographical coverage to draw robust conclusions.

From a methodological perspective, these standard evaluations suffer from several shortcomings. They are deficient not only due to their limited timeframe but also because they lack a counterfactual comparison. The standard evaluation conducted for the French Development Agency in the case of Cafés Lumière does not attempt to compare the treated localities with other similar localities that have not had the same intervention.

Electriciens sans frontière had foreseen this requirement by selecting the 6 treated localities at random from a larger pool of 12 localities. Baseline data on the socio-economic profile was collected in collaboration with FERDI for these 12 localities. However, the use of this survey framework, which is inherently costly, may not have been suitable for 2020, and we will delve into this matter in the fourth article of this series, noting that a second round of the survey in May 2023 is currently being processed. In the meantime, alternative investigative approaches have been explored, combining the utilization of remote sensing data and activity reports from Anka, the operator.

Further reading in the same serie of articles on “Measuring the Impact of Decentralised Electrification Projects”: Using Remote Sensing: Initial Results on the Impact of Cafés Lumière (2/4), Characterisation of the Impacts of Decentralised Electrification Projects on Access to Electricity Using Locality Data (3/4), Characterisation of the Impacts of Decentralised Electrification Projects on Access to Electricity Using Household Data (4/4).