Evaluation Methods to Inform Policy

Moderator

• Lycia Lima, Professor at Fundacao Getulio Vargas and Deputy Director, Centre for Learning on Evaluation and Results (CLEAR Lusophone Africa and Brazil), Brazil

Panellists

• Gilson Pina, National Director for Planning, Ministry of Finance, Cape Verde
• Odilia Gnassingbe, Executive Director, Autonomous Centre for Studies and Capacity Building for Development, Togo
• Timothy Lubanga, Commissioner for M&E, Office of the Prime Minister, Uganda

Policymakers need answers quickly and can’t always wait for a full evaluation to be completed. There is no single methodology for rapid evaluations, so how is rapid evaluation understood in different country contexts? What role does it play? The panel presented examples from Cape Verde, Togo and Uganda.

• Rapid evaluation results have great potential to inform policy. In times of upheaval, policymakers need information to adjust policies mid-crisis, as was shown during the pandemic when rapid evaluations helped to facilitate timely and targeted responses. Their utility also goes beyond crisis contexts: the potential for rapid evaluation results to affect policy decisions is much higher, due to the time constraints under which governments work.
• Rapid evaluations should complement full evaluations, not replace them. There will be gaps in the range of questions a rapid evaluation can answer, and they can’t provide the depth and scope of information of a full evaluation. Managing this requires targeting, planning and awareness on the part of evaluators. Whether to undertake a rapid or regular evaluation depends on the question being asked, and its complexity. While recognising that evaluation happens at the end of a programme cycle, rapid evaluations during implementation can support course correction.
• There is a balance to be struck between rigour and speed. Rapid evaluations are simple in structure and methodology, cheaper to run, and intensive and participatory in their approach. Paying attention to design, reducing scope to essential topics, and promoting collaboration can all support the production. Issues around independence, available resources and capacity require attention.
• Institutionalizing evaluation requires the systematization of evaluative practice and the availability of capacity and resources, more than a legal framework. It requires a structural system to be established, including strong linkages between the aspects involved in public policymaking processes (budgeting, planning and M&E).
• Partnerships are crucial to ensure the availability of resources and capacity. Governments should partner with CSOs, and universities, to augment technical skills for National Evaluation Systems. International organizations can also support where skills and systems are scarce, weak or unavailable within the country. Building strong linkages between the various aspects of policymaking can help to ensure that resources are available, and the evidence fulfils needs.

Conclusion

Rapid evaluations can provide important and timely support to policy decisions within short time frames and with a smaller budget, to complement full evaluations. To institutionalize the practice and ensure a resilient system, the panel recommended: 1) improving linkages between bodies involved in policymaking to ensure resources are available and the evidence supplied meets demand; 2) involving civil society stakeholders more closely; 3) strengthening national capacities, skills and systems.