Evaluation in Fragile Settings

Moderator

• Sarah Longford, Deputy Director, Evaluation, WFP

Panellists

• Rana Sallam, Regional Evaluation Officer, WFP
• Dr. Abdirizak Hassan Mohamed, Senior Advisor, Ministry of Planning, Economic Development and International Cooperation of Galmudug State, Somalia
• Dr. Sory Ibrahima Monekata, Executive Director at Ebenezer International, Mali
• Dr. Thuraya Edelbi, Deputy Head of Planning and International Cooperation Commission, Planning and International Cooperation Commission, Syria

How do we evaluate in fragile settings? How do we overcome the obstacles and challenges? What do evaluative criteria mean in these settings and what other considerations must we take?

• The success of evaluations in fragile contexts relies on adequate local knowledge. Evaluation teams should include people familiar with the context with community ties to enable access and increase trust with interviewees.
• Contextual sensitivities and the lack of reliable data pose a key challenge for evaluations in fragile settings. Recruiting local teams familiar with the country and context, building trust with the interviewers, training them to recognise their biases, and consideration of the gender composition of the interviewers are essential ingredients for conducting evaluation in fragile contexts. Regional and national networks, evaluation associations, CSOs and academic institutions can often help and support the identification and recruitment of local evaluators.
• The participation of citizens and CSOs can play a vital role in evaluations in fragile settings. Their familiarity with the context, community ties and engagement of community champions can play a strong role in data collection, using innovative methods and advancing the evaluation process. However, there is a need to triangulate the data using different methods. Stream A. Stream B Stream C. Evaluating in the midst of global challenges: fragility, inequalities and environment Stream D
• Evaluation in fragile settings comes with its own set of risks. Among others, key challenges include geographical and administrative boundaries and inaccessibility, lack of M&E frameworks and mechanisms which can compromise data collection, or the censorship and control of the government over data and information. National, regional and sectoral reports, as well as data from statistical bureaux and annual surveys, can help bridge data gaps. In order to avoid the risk of data manipulation to attract international aid, triangulation using different sources should be done to validate the reliability of the data.Reports and data from United Nations and other international organizations can often help cross-reference the data collected.
• Evaluations and sectoral studies can provide vital information about fragile contexts. This information empowers citizens, keeps the government accountable and supports advocacy, decision-making, fundraising and awareness, and helps to direct funding where it is most needed.

Conclusion

Evaluations in fragile contexts pose challenges linked to availability of reliable data, inaccessibility due to administrative and geographical boundaries and contextual sensitivities. The success of evaluation in these contexts relies on adequate local knowledge, which can often be sourced through the engagement of local evaluators and CSOs operational in the area and familiar with the communities. Evaluative evidence generated through evaluations in fragile settings can play a powerful role in raising awareness, empowering communities and ensuring government accountability, while at the same time supporting advocacy, decision-making and targeting funding where it is most needed.