[skip to content]

Introduction

In this module you will learn about evaluation, and specifically about the types of evaluation that assesses the achievements of public interventions and uncover the lessons learned which have wider application. Often termed 'summative', 'ex-post', or 'post-hoc' this sort of evaluation looks back over what has changed over the course of an intervention by a public institution, and tries to answer questions such as:

  • What difference has our project, programme or policy made?
  • Who has benefitted and who has been left out?
  • Did our intervention deliver what we expected and hoped it would?
  • Were the resources spent on it justified by the results achieved?
  • What else might account for the changes that have happened?
  • Should we continue doing this sort of thing, and what have we learned to help us do better in the future?
  • Are there lessons from this experience which can help others do better?

Ex-post evaluation is of growing importance for governments, donors, international bodies and those involved in delivering projects and programmes 'on the ground'. It has spawned an industry of researchers, evaluators and consultants who specialise in designing methodologies to answer the sort of questions posed above, and who sell their services to public bodies who want to know the answers. In many cases, the provision of public funds for an endeavour bring with them a requirement for an eventual ex-post evaluation to be carried out – and a concomitant need for data to be collected right from the beginning.

You will hear the term evaluation used in many different ways and applied at different times. This module focuses on the type of evaluation that takes stock and forms judgements. It can be contrasted with other types of evaluation that you may encounter, and which are covered to a greater or lesser degree in a number of other CeFiMS Public Policy and Management courses. You will look in detail at the set of questions that need to be answered once something has been done. As you will see, there are clear connections between different times and types of evaluation, but ex-post evaluation has its own particular problems, contexts and methods.

Learning outcomes

When you have completed this module and its readings, you should be able to:

  • outline how ex-post evaluation informs public policy and management processes
  • set the meaning of ex-post evaluation in the wider context of different ways to assess public intervention
  • distinguish between ex ante, interim, and ex post evaluation, and set ex post evaluation in its proper temporal context
  • specify the differences between public policies, programmes and projects, and the implications of these for their evaluation
  • define a wide variety of evaluation terms and organise them into their purpose, scale, period in time and type of study design
  • choose an evaluation methodology that is relevant to the institutional context and resources available
  • design and carry out evaluations of different types, with an understanding of likely challenges, including identifying and validating impact, disentangling impacts from other influences; the likelihood of absent baselines, inadequate monitoring data, and insufficient research capacity
  • critique evaluation studies in relation to the quality of their design, their conduct and the value of conclusions drawn
  • fit evaluation practice into the real world of politics and the question of what those commissioning really want to know, and want to hear
  • practically assess timescales, changing questions, and when to look for results
  • form a judgement as to whether you yourself would like to consider developing a career as an evaluator
  • join in with networking and exchange amongst the evaluation 'community', and identify differences from broader research.

Study materials

Study guide

You will receive a looseleaf binder containing eight units. The units are carefully structured to provide the main teaching, defining and exploring the main concepts and issues, locating these within current debate and introducing and linking the further assigned readings. The unit files are also available to download from the Virtual Learning Environment.

Textbooks
  • Palfrey C, P Thomas & C Phillips (2012) Evaluation for the Real World. The Policy Press.
  • Bamberger M, J Rugh & L Mabry (2012) RealWorld Evaluation: Working Under Budget, Time, Data, and Political Constraints. 2nd Edition. Sage.
Readings

The Module Reader contains a number of academic articles, case studies and opinion pieces selected to shed light on different aspects of ex-post evaluation, and to provide examples of evaluation in practice.

Virtual learning environment

You will have access to the VLE, which is a web-accessed study centre. Via the VLE, you can communicate with your assigned academic tutor, administrators and other students on the module using discussion forums. The VLE also provides access to the module Study Guide and assignments, as well as a selection of electronic journals available on the University of London Online Library.

Module overview

Unit 1 History, Types and Uses of Evaluation
  • 1.1 Introduction
  • 1.2 Evaluation in Public Policy
  • 1.3 Judgement: Evaluation, Audit, Inspection, Journalism and Academic Research
  • 1.4 Different Types of Evaluation
  • 1.5 Conclusion
Unit 2 Designing an Ex-Post Evaluation: Programme Theory and the Importance of Context
  • 2.1 Ex-Post Evaluation
  • 2.2 Programme Theory: How Interventions are Supposed to Work
  • 2.3 The Importance of Context
  • 2.4 Sources of Information
  • 2.5 Conclusion
Unit 3 Project Evaluation I
  • 3.1 Micro Evaluation – Policies at the Coalface
  • 3.2 Pilots and the Mainstream – Projects as Experiments or Examples of Basic Delivery
  • 3.3 Projects in Context
  • 3.4 Drawing Conclusions and the Question of Attribution
  • 3.5 Project Evaluation as a Laboratory – the Question of Replicability
  • 3.6 Time to Get Going: Scoping the Evaluation
  • 3.7 Conclusion
Unit 4 Project Evaluation II
  • 4.1 The Case Studies
  • 4.2 Purpose and Context
  • 4.3 Theory of Change
  • 4.4 Evaluation Design and Methodology
  • 4.5 Results and Conclusions in the Reports
  • 4.6 Conclusion
Unit 5 Evaluation Methods: Quantitative, Qualitative and Mixed
  • 5.1 Introduction
  • 5.2 Quantitative Evaluation Methods
  • 5.3 Qualitative Evaluation Methods
  • 5.4 Mixed Method Evaluation
  • 5.5 Conclusion
Unit 6 Programme Evaluation I
  • 6.1 Programmes and Their Evaluation
  • 6.2 Evaluation Criteria
  • 6.3 The Specification of Impacts
  • 6.4 Conclusion
Unit 7 Programme Evaluation II
  • 7.1 The Case Studies
  • 7.2 End-of-Life Care in Nursing Homes
  • 7.3 The Tuungane Community-Driven Reconstruction Project
Unit 8 Policy Evaluation, and Being an Evaluator
  • 8.1 Introduction
  • 8.2 Policy Evaluation
  • 8.3 Policy Evaluation in a Development Context
  • 8.4 Multinational Policy Evaluation
  • 8.5 Maximising the Usefulness of Evaluation
  • 8.6 Conclusion
  • 8.7 Post Script: Being an Evaluator
  • 8.8 Glossary of Standard Evaluation Terms

Tuition and assessment

Students are individually assigned an academic tutor for the duration of the module, with whom you can discuss academic queries at regular intervals during the study session.

You are required to complete two Assignments for this module, which will be marked by your tutor. Assignments are each worth 15% of your total mark. You will be expected to submit your first assignment by the Tuesday of Week 5, and the second assignment at the end of the module, on the Tuesday after Week 8. Assignments are submitted and feedback given online. In addition, queries and problems can be answered through the Virtual Learning Environment.

You will also sit a three-hour examination on a specified date in September/October, worth 70% of your total mark. An up-to-date timetable of examinations is published on the website in April each year.

Module sample

Click on the link below to download the module sample document in PDF.