About
More than evaluation consulting. Beyond capacity building.
The program reaches deeper into an organization than traditional evaluation consulting, and it is far more participatory, on both sides, than typical capacity-building models. The model is designed to incorporate outcomes sensitivity through the following impact cascade:
-
01
Evaluation
Establishes the systems and standards for recursive improvement, and the baseline measurements that change is rated against.
-
02
Programs
Evaluation results inform program changes, and drive improvements in implementation fidelity and measurability.
-
03
Outcomes
The program team defines implementation strategies from the recommendations, and reports results, roadblocks and community concerns.
-
04
Recursion and integration
Traces the processes that translate metrics to outcomes and builds cross-cutting improvement strategies that feed into the next evaluative cycle.
Why recursion and integration exists. Studies of existing models find a weak connection between capacity-building services and final program outcomes. Existing models, due to their reliance on time-determined fees and discrete deliverables, abdicate their accountability to outcomes by detaching from practice context. Recursion and integration explicitly does the opposite: qualitative analysis traces each evaluation and program input through to its effect on downstream outcomes. This keeps our practice accountable to outcomes rather than to deliverables.
An outcomes-oriented synthesis.
Program evaluation is inaccessible and underleveraged. The evidence says evaluation works best through long-term collaboration that builds trust, deepens context and shepherds adoption. In the current market, those carry impossible costs for small and mid-sized organizations.
Implementation science
The study of how new practices take hold in real programs, and how to shepherd adoption while ensuring that they are aligned with target needs and capabilities.
Evaluation capacity building
The practice of making evaluation, and the use of its findings, an ordinary part of a program's operations.
-
Learning from the process
Process use names the learning that comes from taking part in an evaluation, beyond anything in its findings.
-
Capacity by design
Evaluation capacity building is defined as a field: designed practice rather than a by-product, with the use of evaluation built into what capacity means.
-
Capacity lives in culture
A multidisciplinary model ties evaluation capacity to organizational learning and culture, and sets out the conditions for practice that lasts.
-
A framework for what shapes implementation
The Consolidated Framework for Implementation Research organizes the factors that help or hinder putting evidence into practice.
-
Dose matters
A two-year cluster-randomized trial finds dose-dependent gains across four capacity scales, greatest where technical assistance hours were highest.
-
Organizations, not individuals
Instrument validation establishes that leadership, resources and learning climate decide whether individual knowledge ever becomes mainstreamed practice.
-
Doing is not using
The capacity to do evaluation is separated from the capacity to use it: methodological skill on one side, the organizational conditions that let findings have influence on the other.
-
A habit of mind
Evaluative thinking is named as the lasting disposition that evaluation capacity building aims to instill.
-
Implementation support becomes a field of research
Two intermediary organizations, one in the USA and one in Ireland and Northern Ireland, begin studying the role and competencies of the people who support implementation.
-
Repetition and duration drive results
A replication trial finds no significant effect in year one and small significant improvement after a second-year quality improvement cycle.
-
Competencies, described
UNC's Collaborative for Implementation Practice publishes the Implementation Support Practitioner Profile and Practice Guide which set out fifteen core competencies in three domains: co-creation and engagement, ongoing improvement, and sustaining change.
-
Relationship, not training
Evaluation practice persists years after an intervention, contingent on an ongoing relationship with the evaluator rather than the initial training.
-
The role comes into focus
A white paper from The Center for Implementation complements previous work with ten core competencies for implementation support practitioners and five functions of their role: connect, understand, inspire, enable and transform.
Where the two fields meet
Whether they be survey methods, practice changes, data collection techniques, metric integrations, or monitoring platforms, comprehensive evaluation surfaces myriad new processes that should be shepherded with care and intentionality. No existing model offers program improvements with the care and sustainability delivered by evaluation capacity building grounded in the practice of implementation science.