Program Theory in Public Administration

Key Concepts and Terms

• Definition and scope. Program theory focuses on the social model underlying a policy program. For instance, in a program geared to job training for ex-convicts, the social model which provides a rationale for the program may be one with a variety of causal variables related to reduction of recidivism as the dependent variable. "Program theory" is thus a meta-theory with innumerable particular instances of it (ex., a theory of recidivism). Weiss (1998) defines program theory as 'theories of change' relating programs to desired outcomes.

• Logical framework analysis. Program theory is closely associated with "program logic" and the "logical framework model." The Logical Framework (Logframe) model was developed by USAID in 1969 for evaluating programs in international development. Logframe became widely used in the next two decades and in 1997 was adopted by the World Bank for similar purposes (World Bank, 2000). A central aspect of Logframe is the "results framework," in which the analyst reasons backward from the desired results. For instance, for the desired result of "poverty reduction," a prior result would be "rapid, sustained growth rate," which in turn might have several priors, one of which would be "stable economic foundation," the priors for which would be "stable, regulated financial sector," "stimulation of private investment," and "establishment of export mechanisms." Eventually the analyst reasons backward to specific investment programs that USAID or the World Bank might wish to support as part of the logical framework model.
  • Logic modeling is essentially the same concept as logical framework analysis. As articulated by the Kellogg Foundation (2004), program impacts are caused by a prior chain, which in reverse chronological order are program outcomes, program outputs, program activities, and program resources/inputs. When presenting a proposed program, each link in the chain must be reasoned: (1) if certain resources are provided, then the program can undertake certain activities; (2) if activities occur, then there will be certain outputs (direct results); (3) if outputs occur, the outputs will lead to certain outcomes (changes in attitudes, behaviors, knowledge, skills, status, or level of functioning); and (4) these outcomes will lead to the end impacts (organizational, community, and/or system level changes) desired. Thus, "The purpose of a logic model is to provide stakeholders with a road map describing the sequence of related events connecting the need for the planned program with the program’s desired results" (Kellogg Foundation, 2004: 3).
    • Outcome hierarchies. Duignan (2004) and others have noted that the activities-outputs-outcomes-impacts chain may be oversimplified. Reality may be better described as a system of outcome hierarchies. "An outcome hierarchy is a structured set of all of the important intermediate outcomes which lead through to a final outcome being sought by an organisation or group of organisations" (p. 1). Duignan's "outcome theory" calls attention to the possibility of multiple levels of cascading outcomes, multiple outcomes for any given domain, cross-over effects between outcomes and levels, and the fact that issue domains transcend program organizations, which themselves operate at different levels. In general, Duignan approach to "strategic evaluation" promotes the thesis that program evaluation should be reconceptualised as being about meeting the evaluation knowledge needs of an issue domain and should not be primarily about individual program evaluations.

  • Relation to strategic planning. Logical framework analysis is a planning process as well as a method of developing a causal theory. Fear (2006) has noted the close correspondence of program theory with strategic planning for this reason. Key elements of Logframe planning include formulation of strong impact statements, client-centered program design, use of performance indicators, participatory involvement of stakeholders, risk assessment, and openness/accountability.

• Relation to program evaluation. Program theory is advanced as an alternative to narrowly focused program evaluations which may measure inputs and outputs (efficiency), without ever measuring the more important relation of causes to desired effects (effectiveness). Chen (1990) defines program theory as 'theory-driven evaluation.'
  • Impact theory. Rossi, Freeman, and Lipsey (1999) elaborated a program theory framework for program evaluation. In this framework, organizational plan variables lead to organizational capacity leads to service utilization by the target population, leading in turn to desired change outcomes as predicted by impact (program) theory. Impact theory variables are the change theory (program theory) for the particular policy/program at hand. Organizational plan variables include program resources, personnel, administration, and organization. Service utilization variables include outreach, access, and agency-target relationships. A comprehensive program evaluation must be a three-fold assessment, looking at impact theory, the organizational plan, and service utilization.

• Relation to implementation theory. Scheirer (1987) views implementation theory as a complement to program theory, defining implementation theory as a focus on variables related to program delivery mechanisms (inputs and outputs, measuring efficiency of program delivery). Scheirer urges program evaluators to employ both program theory and implementation theory perspectives.

  
  

Assumptions

• Specific change theory required. In any particular analysis, one cannot have a "program theory framework" in general without also having a specific, testable change theory related to the subject at hand. For instance, one cannot have a "program theory framework" to analyze a job training program for ex-convicts without having a specific, testable change theory of recidivism (assuming reduction of recidivism is the intended primary program impact).

• Minimal unintended consequences. Program theory is only as strong as human logic. Particularly when applied a priori, as in program justifications submitted to funding agencies, insufficient recognition may be given to the fact that unintended consequences of rational actions may lead to different, even opposite impacts.

• Issue interaction may be neglected by program theory, which tends to isolate reasoning to look at a single program issue in isolation from the development of other issues in the environment. Trends in other issue areas may negate the program logic studied by the program theory analyst.

• Neglect of political issues. Though not necessarily intrinsic to program theory, like other rational approaches, program theory tends to focus on "rational" causal factors and neglect "irrational" factors of a political, social, or psychological nature. In this way, program theory may oversimplify.

Illustrative Hypotheses

• Program theory itself does not generate testable hypotheses or predictions, but the particular change theories enacting program theory do.

Bibliography

• Bickman, L. (1987). The function of program theory. New directions for Program Evaluation 33: 5-18.
• Chen, H. (1990). Theory-driven evaluations. Newbury Park, CA: Sage Publications.
• Duignan, P. (2004). Principles of outcome hierarchies: Contribution towards a general conceptual framework for outcomes systems. Parker Duignan Ltd. "Strategic Evaluation" website. Retreived 1/8/07 from http://www.strategicevaluation.info/documents/122.htm.
• Fear, William J. (2006). Programme evaluation theory: The next step towards a synthesis of logic models and organisational theory. Paper presented to Evaltalk, 31 December 2006. American Evaluation Association Discussion List; evaltalk@bama.ua.edu.
• Finney, J. W. and R. H. Moos (1989). Theory and method in treatment evaluation. Evaluation and Program Planning, Vol. 12: 307-316.
• Kellogg Foundation (2004). Logic Model Development Guide. Battle Creek, MI: W. K. Kellogg Foundation. Retrieved 1/8/07 from http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf.
• Rossi, P. H., Freeman, H. E., and Lipsey, M. W. (1999). Evaluation: A Systematic Approach. 6th ed. Thousand Oaks, CA: Sage Publications.
• Scheirer, Mary Ann (1987). Program theory and implementation theory: Implications for evaluators. . New directions for Program Evaluation 33: 59-76.
• Shadish, William R., Jr. (1987). Program micro- and macrotheories: A guide for social change New Directions for Program Evaluation 33: 93-109.
• Weiss, Carol H. (1998). Evaluation: Methods for Studying Programs and Policies. 2nd ed. Upper Saddle River, N.J.: Prentice-Hall.
• World Bank (2000). The LogFrame Handbook: A logical framework approach to project cycle management. Washington, DC: World Bank. Retrieved 1/8/07 from http://www1.worldbank.org/education/adultoutreach/Doc/LFhandbook.pdf.