Artificial intelligence and the opportunities it brings to evaluation

Moderator

• Patrizia Cocca, GEI/World Bank Group, Communications and Knowledge Management Lead

Panelists

• Anish Pradhan, UNDP IEO, ICT Specialist
• Devika Lakhote, 3ie, United States of America, Data Scientist
• Lingrong Ying, Shanghai University of International Business and Economics, Shanghai, China, Research Assistant 
• Marco Segone, Evaluation Office, UNFPA, Director
• Thomas Kelly, 3ie, United States of America, Director

Discussion Themes

• AI has application in all steps of the evaluation process—from writing terms of reference and proposals, to designing methods and forms, data synthesis, drafting questions, text data synthesis, data analysis, conversion and first draft development of evaluation reports.
• Humans are accountable, while machines are not. It is important to consider and address the ethical implications of AI when incorporating it into evaluation work. This may look like including an ethical AI use clause in contracts with consultants, clear and transparent documentation of AI models, the creation and use of an AI disclaimer on evaluation products, allowing for human verification and oversight of work undertaken by AI, and the thoughtful prioritization of data protection measures when using AI tools.
• There are additional challenges related to the use of AI that require solutions, including the energy cost and environmental impact of its development and use, bias in training data, black box problems when determining how the AI technology is making its decisions in its outputs, the opportunity for factual accuracy issues, and opaque data sources.
• AI is an ongoing investment. Development of this tool should be intentional in experimenting and adapting it to changes in context and technological progress.