Plenary 3

Moderator

• Oscar A. Garcia, Director, Director, IEO, UNDP and Chair, United Nations Evaluation Group (UNEG)

Panellists

• Andrea Cook, Director, Evaluation, WFP
• Juliet Parker, Director, Active Learning Network for Accountability and Performance (ALNAP)
• Alison Evans, Director-General, IEG, World Bank Group
• Isabelle Mercier, Director International Assistance Evaluation, Global Affairs Canada
• Dr. Angelina Mattijo-Bazugba, Associate Professor of Social Policy, and Dean/ Director of National Transformational Leadership Institute, University of Juba, South Sudan

Pivoting evaluation and developing new approaches to address global crisis needs and resilience. The role of evaluation in changing approaches.

• Evaluations support transition in and out of humanitarian crises, and stimulate looking forward. International organizations need to focus not only on what they can do, but what they are doing jointly with other humanitarian organizations. This increases the role of evaluations carried out by different partners.
• Crisis evaluations propel real-time learning. While learning within the humanitarian context is challenging and incremental, it provides an opportunity for adaptation and change, which is constantly happening in humanitarian settings.
• Caution needs to be exercised when generalizing crisis situations. Evaluating during crisis and evaluating crisis responses should be clearly differentiated. Evaluative evidence generated during a crisis should be nuanced from that generated in a normal development setting.
• In many countries, crisis is the normal way of doing things. For example, South Sudan separated from Sudan in 2011, then a conflict started in 2013, another in 2016 and again in 2018. The country is currently in the transition phase to implement the peace agreements. Evaluations in these settings help to identify what works and why, so that legislation can be improved with information and timely evidence for policymakers.
• Learning on operating within a crisis context has been slow. While there has been some progress at technical level, a lot more needs to be done to improve accountability towards vulnerable populations. Adaptations at country level are not often commensurate with adaptations in practice at the sector level, and different levels of government, owing to resistance from central structures to change, especially where structural reforms are required.
• COVID-19 was a real push for the evaluation sector to real-time evaluating and learning processes which made faster movements to respond to the crisis. Modular approaches that define high-level questions and learning priorities can promote real-time learning.
• Evaluators need a big dose of humility as we don’t have the right answers all the time, it is really important to understand how we fit into the learning process and be flexible enough to integrate into it.
• There is need for clarity on where and when evaluation can play a role in the real‑time learning and when and where it can contribute with strategic elements that only evaluation brings.
• Synthesis from the stock of knowledge and evaluations can play a key role in the learning process and provide timely evidence for informed decision-making. Having an open discussion within the organization and stakeholders is critical for setting the right direction.
• Trust between evaluators and the development community is essential for evidence‑based informed humanitarian programme funding decisions. Unless that trust is built, the decision-making process will not be effective.
• Evaluation should be mindful of the context, and be realistic and not generalist. It is critical to be flexible, open minded and sensitive to the local culture and contexts.
• Evaluations in fragile settings should be cognizant of the challenges posed when operating in difficult situations. While evaluations need robust and credible data, it is important to be humble and open to deep listening and non-traditional data sources and ways of working. Evaluators need to be bold and innovative and stop recommending the same things that have been recommended for years.
• Evaluation results are not always translated into policy changes, sometimes small incremental changes are sufficient, and evaluators should consider that and not always aim for a higher level of structural reform. There is a need to carefully consider ethical issues when evaluating in fragile settings.

Conclusion

It is important to make a distinction between evaluating during crisis and evaluating crisis response.More than real-time evaluation, real-time learning should be prioritized in fragile contexts.Methodological innovations and synthesis of knowledge from past evaluations can often play a central role in the learning process and provide timely evidence for informed decision-making.