Theory-Based Approaches to Evaluation: Concepts and Practices
6. Developing Theories of Change
There is considerable literature (see References) on developing theories of change. Much of it is written for and by evaluators, who often have to develop such theories when they undertake evaluations. A preferred and increasingly common scenario is for intervention designers and managers to develop a theory of change during the initial design of the intervention as an aid to intervention planning and planning for performance measurement. This is good practice for results-based management and is encouraged where appropriate. Section 10.0 discusses these uses of theories of change.
When evaluators are retroactively developing a theory of change, the following sources of information should, at a minimum, be consulted:
- Key intervention documents (e.g., Memoranda to Cabinet, Intervention Terms and Conditions, and planning documents)
- Relevant literature review (e.g., prior evaluations and social science research)
- Intervention managers
- Subject-matter experts
- The program’s logic model
The development of a theory of change generally involves three steps:
- Developing a logic model with clear results chains and explicit causal links (a basic theory of change)
- Identifying assumptions and risks underlying the theory of change
- Identifying other contextual factors associated with the results chain. The result is a refined theory of change.
Developing a Basic Theory of Change
The development of a basic theory of change involves identifying an intervention’s activities, outputs, and the sequence of outcomes needed for the expected results to occur. This often involves developing a logic model with reasonable results chains and describing what the arrows or other connections in the results chains imply.
For example, Figure 2 provides an example of a basic theory of change for an intervention aimed at building effective results-based management (RBM) in an organization. The left-hand side of Figure 2 shows a results chain for the intervention, which includes providing training and guidance on measuring and monitoring results, which leads to improved systems, capacity and institutionalization of RBM, which leads to the use of results information to inform decision making; which leads, ultimately, to the improvements in organizational program performance. The right-hand side of Figure 2 identifies the causal links between the outputs and outcomes (i.e., what “happens” in the gray arrows that link the outputs and outcomes). An outline or summary of the intervention’s basic theory of change appears at the bottom of Figure 2.
The blue arrows, which flow backward from the ultimate outcome through to outputs, indicate that although the results chain is pictured as a linear process, there is normally feedback between the different stages in the results chain. For example, the availability of more and better results information can lead to the need for more training and guidance. Similarly, an increase in the use of results information can lead to an increase in demand for good results information.
In developing theories of change, it is also useful to identify the degree of influence the intervention has in terms of its causal link, the degree of control the intervention has in ensuring that the causal link is realized. Figure 2 illustrates this by labelling the causal links as being directly influenced [DI], influenced [I] or outside the influence [O] of the intervention.
Figure 2. A Basic Theory of Change for Enhancing Results-Based Management in Organizations
The outline of the intervention theory of change: By providing information, training and hands-on assistance in RBM, organizations will build up their RBM capabilities and related systems and use the results information to inform their decision making, leading to more effective and efficient programs.
- [C] - control
- [DI] - direct influence
- [I] - influence
- [O] - outside of influence
Refined Theories of Change: Identifying Assumptions, Risks and External Factors
Refined theories of change go beyond the basic theory to identify the assumptions behind the various causal links in the results chain and the risks associated with those assumptions. This helps ensure that the theory of change explains what conditions have to exist for each causal link to be realized (i.e., for A to lead to B). Figure 3 sets out a refined theory of change for the RBM example, showing the assumptions and risks behind the causal links identified in the basic theory of change. Alternatively, instead of representing the assumptions as done in Figure 3, they can be presented as a list of premises that need to be tested. This is the approach that Leeuw, Gilse and Kreft (1999) used.
As with the basic theory of change, it is useful in refined theories of change to identify the degree of influence that an intervention has over these assumptions and risks. However, as noted in Figure 3, a new level of influence (control [C]) is introduced to denote areas where the intervention should be able to effectively control a particular condition (e.g., the production of outputs).
It is also important to identify the significant external factors, or contextual factors, that might influence the intended outcomes. These external factors are generally situations or events that are outside the direct control of the intervention to influence, manage or prevent. These can be illustrated as part of the theory of change. In Figure 3, for example, external influences, identified on the left-hand side of the results-chain, may include requirements of funding agencies, negative experiences with past management initiatives or management trends among peers to improve intervention monitoring and evaluation.
Setting out the assumptions, risks and external influences helps describe both the intervention and the context in which it is operating. Defining assumptions, risks and external factors at the intervention design stage can help identify additional activities that the intervention may wish to undertake as part of its risk management. This may allow an identified external influencing factor over which the intervention previously had no influence to be converted to an internal risk that can, to some extent, be mitigated. This expanded results chain/theory of change can be useful as a framework both for evaluation purposes (where the theory of change would be tested with empirical data from the intervention to determine whether the theory is working) and for enhancing reporting on the performance of the intervention (see Section 10).
Discussions on developing theories of change and examples are available on the Theory of Change Community website.
Figure 3. A Refined Theory of Change for Enhancing Results-Based Management in Organizations
- [C] - control
- [DI] - direct influence
- [I] - influence
- [O] - outside of influence
- Chen, H.-T. (2003). Theory-driven approach for facilitation of planning health promotion or other programs. Canadian Journal of Program Evaluation, 18(2), 91–113.
- Connell, J. P., & Kubisch, A. C. (1998). Applying a theory of change approach to the evaluation of comprehensive community initiatives: Progress, prospects, and problems. In K. Fulbright-Anderson, A. C. Kubisch & J. P. Connell (Eds.), New approaches to evaluating community initiatives: Vol. 2, Theory, measurement, and analysis. Washington, DC: The Aspen Institute.
- Funnell, S. (2000). Developing and using a program theory matrix for program evaluation and performance monitoring. New Directions for Evaluation, 87, 91–101.
- Funnell, S., & Rogers, P. (2011). Purposeful program theory: Effective use of theories of change and logic models. San Francisco, CA: Jossey Bass.
- Goertzen, J. R., Hampton, M. R., & Jeffery, B. L. (2003). Creating logic models using grounded theory: A case example demonstrating a unique approach to logic model development. Canadian Journal of Program Evaluation, 18(2),115–138.
- Hatry, H. (2006). Performance measurement: Getting results (2nd ed.). Washington, DC: The Urban Institute Press.
- Leeuw, F. L. (2003). Reconstructing program theories: Methods available and problems to be solved. American Journal of Evaluation, 24(1), 5–20.
- Leeuw, F. L., Gilse, G. H., & Kreft, C. (1999). Evaluating anti-corruption initiatives: Underlying logic and mid-term impact of a World Bank Program. Evaluation, 5(2), 194–219.
- Mason, P., & Barnes, M. (2007). Constructing theories of change. Evaluation, 13(2), 151–170.
- Organizational Research Services. (2004). Theory of change: A practical tool for action, results and learning (PDF – 369 KB). Seattle, WA: Annie E. Casey Foundation.)
- Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Thousand Oaks, CA: Sage Publications Inc.
- Rogers, P. (2008). Using program theory to evaluate complicated and complex aspects of interventions. Evaluation, 14(1), 29–48.
- Weiss, C. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In J. P. Connell, A. C. Kubisch, L. B. Schorr & C. H. Weiss (Eds.), New approaches to evaluating community initiatives: Vol. 1, Concepts, methods, and contexts (pp. 65–92). Washington, DC: The Aspen Institute.
- Date modified: