Trainer: Stephen Porter
Level: Beginning/intermediate
Language: English
Workshop short description:
Sustainable development is more likely to be effectively achieved by building sustainable organizational evaluation systems. Unsustainable organizational evaluation systems are less likely to contribute to development as they will not be able to:
• develop capacity to value changes within complex adaptive systems;
• develop mechanisms to support the process use of their evidence;
• support learning on what’s working and are we doing the right things; or
• develop networks to help address complex problems.
The need to think about evaluation systems, their effectiveness, and their contribution toward sustainable development has become pertinent in recent years with the establishment of increasing numbers of organizational evaluation systems. Further, the Sustainable Development Goals require country-led evaluation as part of follow-up and review processes (UN, 2015).
This workshop provides participants with an understanding of
1. Issues that reduce sustainability;
2. Developing use orientated organizational evaluation systems;
3. Tailoring Quality expectations to the context; and
4. The role of networks in sustainable evaluation systems.
The workshop argues that organizational evaluation systems become sustainable and respond to common issues through being networked, use-focused, and by undertaking processes aligned with agreed-upon quality standards.
Workshop detailed description:
Rationale: Organizational evaluation systems that are sustainable are an important component of national evaluation systems. They are likely to be able to help accumulate knowledge and be well placed to challenge underlying issues that exacerbate exclusion. Evaluation systems become sustainable not only when they are of quality, but also implemented to respond to issues that reduce their sustainability. This argument is elaborated in the following article, which is also the basis for this workshop: Porter, S., & Hawkins, P. (2019). Achieving sustainability through sustainable organizational evaluation systems. In G. Julnes (Ed.), Evaluating Sustainability: Evaluative Support for Managing Processes in the Public Interest. New Directions for Evaluation, 162, 87– 101.
Teaching strategy: techniques involving participation and case studies led by experienced facilitator.
Type of participant: Government, Civil Society or Private Sector involved in developing an organizational evaluation system.
Topics Covered: This workshop is split into four main segments.
1. Common issues that reduce evaluation system sustainability;
2. Developing use orientated organizational evaluation systems;
3. Tailoring Quality expectations to the context; and
4. The role of networks in sustainable evaluation systems.
Issues that reduce Evaluation System Sustainability
Unsustainable evaluation systems include those that fail and those that struggle with denuded capabilities, consequently producing lower quality and less useful evaluative evidence. In such situations, evaluation ends up disconnected from decision-making and is not directed toward ongoing improvement to meet current needs. Five issues that affect organizational evaluation are discussed in this section of the workshop:
1. An aversion to the reality of results;
2. Disfigured accountability;
3. Ritualization;
4. Empty rhetoric; and
5. Censorship.
Developing use orientated organizational evaluation systems
Many evaluators highlight challenges in evaluation use. Improving evaluation utilization is a task that is often far larger than the individual evaluator, it is a systems issue. It is important for evaluation commissioners and managers to move towards systematic evaluation practice that meets the demands of the organization and increase utilization of evaluation findings. This section considers:
• Evaluation as a political act that requires cooperation among multiple stakeholders
• The role of incentives in an evaluation system
• How an evaluation system can challenge underlying norms and culture in a way that can be heard
Tailoring Quality expectations to the context
The third area covered is on evaluation quality. Sustainable evaluation system must have processes and products of good quality that work in the context. This course provides illustrations of different organizational evaluation systems, which often define their own typologies of evaluation, interpretations of quality standards, methods, and quality review mechanisms depending on their own institutional demand for evaluation. This section considers:
• Evaluation typologies and their impact upon evaluation choices; and
• Exploration of how quality standards help to direct evaluation practice towards or away from sustainability.
The role of networks in sustainable evaluation systems
Evaluation systems that build coordinating, cooperating, and collaborative networks are more likely to be sustainable and contribute toward sustainable development. In this course, it is argued that a good evaluation network can function to continually strengthen connections of evaluative thinking and practice, making evaluation less scary and support ongoing reflection and discussion to contribute to adaptive improvement. This section considers:
• Networks and shared goals in an evaluation system;
• Networks and evaluative thinking; and
• Networks and responsiveness to demand.
Learning outcomes:
Upon completing this course participants will be able to:
• Describe issues that reduce evaluation system sustainability;
• Understand the different types of demands and incentives that drive the use of evaluation;
• Give example of types of quality control and assurance processes in evaluation systems; and
• Describe how networks support an evaluation system.