Article published origionally at the Guardian

Image: Internews Europe

A new study challenges the traditional assumption that experts are best placed to evaluate how well people could cope with and adapt to an emergency

The concept of “resilience” is taking development and humanitarian sectors by storm. Huge amounts of finance are being channelled into “resilience-building” activities, aimed at supporting people and communities to deal more effectively with climate extremes, financial shocks and the many other risks that threaten lives and livelihoods.

Given the pressure on NGOs, governments and donors to demonstrate value for money and support the right people and activities, the race is on to find the best ways of measuring resilience.

Normally, the first step in designing a method for resilience measurement is to consult a group of experts, who consider the assets and capacities that make a household robust. They typically come up with a number of indicators, which can consist of anything from household income and child nutrition to social networks and access to financial capital. Each indicator and characteristic is then mashed together and weighted, often resulting in a single overall score.

While useful, these approaches make a critical assumption: that experts are best placed to evaluate someone else’s resilience. In a new paper for the Overseas Development Institute, we challenge that assumption and propose an alternative approach that has been largely overlooked, but may help to address many of the challenges associated with traditional ways of measuring resilience.

The approach is called subjective resilience, and it starts from the premise that most people have a good understanding of the factors that contribute to their own ability to cope with and adapt to emergencies.

People are asked to consider the factors contributing to their livelihoods and judge how resilient they consider their household to be to given threats. They are also asked to suggest ways to enhance their resilience.

To test the approach, a consortium of international NGOs and research organisations under the Global Resilience Partnership teamed up with Twaweza, an east African movement working to give citizens more power, to speak to 1,300 households across Tanzania, asking people to evaluate their ability to cope with and adapt to the risk of future flooding.

The method used to collect data was also quite unique. Face-to-face interviews were done with people in households from the shores of the Indian Ocean to the Serengeti. Each person was then given a mobile phone and a solar charger, and every few months a call centre rang the same people to ask a series of questions. The answers were then analysed to give accurate and nationally representative information and the whole process was cheaper and easier than traditional surveys.

Given that surveys relating to subjective resilience tend to be shorter than objective ways of measuring resilience (which can often be more than 100 questions long), the approach lends itself to data collection of all kinds across Africa.

Initial results suggest that many of the factors that experts associate with a person’s resilience may not be as important as they had assumed. Factors such as level of education, livelihood, or age have some effect on people’s perceived resilience, but relationships are not as strong as one might traditionally expect. Other factors, like gender, have no apparent impact on evaluations of household resilience, even among female-headed households (typically considered to be one of the most vulnerable groups). Unsurprisingly, the strongest predictor of resilience is wealth and level of income.

While the research is in its early stages, and more work needs to be done to test and develop subjective methods, what we have seen so far raises questions as to whether the indicators we equate with resilience are the right ones.

Above all, subjective assessment has the potential to radically change the way that we track resilience and hold governments and civil society to account.

It allows for a bottom-up way of judging the effectiveness of resilience-building initiatives based on the perspectives of the people that matter most: those who are vulnerable and receiving support. One interesting finding from the Tanzania survey was that those who received early warning information about floods rated themselves as far more resilient than those who hadn’t. This suggests that the considerable efforts to support dissemination of weather forecasts and alerts in east Africa may be paying off.

This approach could be used to assess the effectiveness of investments and projects from the perspective of the people benefiting from them. If we listen to their collective voices, we might be better able to hold NGOs, businesses and governments to account.

The same tools could also be applied to evaluate national or international resilience-building initiatives: a difficult feat so far. If international commitments, such as the sustainable development goals, are taken seriously and policies implemented to achieve them, then it is only reasonable to expect a marked difference in how resilient local people perceive themselves to be.

It is clear that subjective assessments are not without weaknesses. People aren’t always aware of all the factors that make them resilient, and some may choose to answer questions inaccurately in order to gain from the outcome of the survey, particularly if there is an assumption that the survey is linked to NGO or government assistance. However, many of the biases can be accounted for by thorough research design and by providing clear information on how the data will be used. Confidence can be taken from the success of similar approaches in other fields, such as subjective wellbeing.

Ultimately, collecting information on subjective resilience isn’t meant to replace traditional measurement entirely. Rather, bottom-up subjective methods should be used alongside objective methods, helping to capture the components of resilience that are difficult to observe and allow people’s perspectives to be heard.

If we can get the measurement process right, this will be an important step forward in gaining a more holistic understanding of what it takes for a household to be resilient to the many risks and threats it faces.