Service evaluation and programme evaluation

• Explain the purpose of service evaluation and how it differs from audit, QI and research.
• Frame useful evaluation questions about reach, process, outcomes, experience, equity and value.
• Choose proportionate mixed methods for evaluation.
• Recognise governance, confidentiality and ethical obligations in non-research evaluation.
• Use findings to make defensible decisions about continuation, adaptation or withdrawal of a service.

Service evaluation examines an existing service, pathway or programme to understand how it operates and what it is achieving in its local context. It may ask about reach, activity, quality, outcomes, patient experience, workforce effects, cost or equity. It usually does not assign treatment for the purpose of generating generalisable new knowledge.

The label matters because governance routes differ, but classification should follow the design and purpose rather than the team's preferred paperwork. If uncertain, use the HRA decision tools and local R&D or governance advice.

A good evaluation is decision-linked: what decision will the evidence inform? Continue the service? Change eligibility? Increase staffing? Stop an ineffective component? Without a decision question, evaluation can become descriptive data collection with no endpoint.

Start with the programme theory: what resources and activities are expected to produce which outputs and outcomes, for whom and under what conditions? A simple logic model can link inputs, activities, outputs, short-term outcomes and longer-term outcomes.

Then frame a small number of evaluation questions. Common domains include reach, fidelity, experience, outcomes, equity, unintended consequences and cost or resource use. A new virtual clinic might improve access overall while disadvantaging patients with digital exclusion; both findings matter.

Specify the comparator where appropriate: before/after, eligible non-users, another site, planned trajectory or national benchmark. The comparator should fit the decision, not imitate a research design unnecessarily.

Routine administrative and clinical data can describe activity, waiting times, outcomes, safety events, attendance and resource use. Define each metric operationally and understand changes in coding or data completeness over time.

Before/after comparisons are useful but vulnerable to secular trends, seasonal variation and concurrent interventions. Use time-series displays where possible rather than relying on two summary points. Stratify important outcomes by relevant population groups to detect inequitable access or effect.

Avoid claiming causation stronger than the design supports. “Admissions fell after implementation” is not the same as “the service caused the fall in admissions.”

Interviews, focus groups, open-text surveys and observation can explain why a service is or is not working. Sampling should seek relevant perspectives rather than simply the easiest participants. Include people who disengaged, declined or experienced barriers where feasible; otherwise the evaluation may over-represent satisfied users.

Use a clear topic guide, obtain appropriate consent or participation agreement, protect confidentiality and document the analysis method. Even a small rapid evaluation should show how themes were derived rather than selecting memorable quotations.

Qualitative findings are especially valuable for identifying mechanisms, burden, acceptability and unintended consequences that routine metrics cannot show.

Before creating a new survey, ask whether an existing validated measure fits the question. Keep surveys short enough to complete, pilot wording with intended respondents and avoid leading or double-barrelled questions. Response rate and non-response bias should be considered when interpreting results.

A satisfaction score can conceal important variation. Pair closed questions with targeted open-text prompts and demographic or service variables only when these are relevant and can be handled safely.

Do not treat a staff survey as anonymous if the combination of role, site and free-text makes individuals readily identifiable.

Not every evaluation needs formal health-economic analysis, but every service consumes resources. Track staff time, consumables, digital costs, opportunity cost and downstream effects where these are material to the decision.

Cost saving should not be asserted merely because an activity count falls. A clinic may reduce hospital visits while transferring workload to primary care or unpaid carers. Evaluation should look across the pathway when possible.

Value is broader than cost. A more expensive service may be justified by improved access, safety or patient outcomes; equally, a popular service may not represent good value if it duplicates existing provision.

Service evaluation is usually outside the UK Policy Framework definition of research, but this does not remove duties relating to confidentiality, data protection, proportionality, fairness and avoidance of harm. HQIP's updated ethics guidance is particularly important here: QI, audit and service evaluation should still be scientifically valid, equitable, valuable and have a favourable benefit-risk balance.

Register the evaluation through the local route. Clarify whether patient contact, use of identifiable information, linkage, recording or publication triggers additional approvals. If the project changes into an activity designed to generate generalisable knowledge, reclassification may be needed.

When in doubt, seek prospective advice; retrospective attempts to justify a chosen label create governance risk.

A health board introduces a same-day frailty assessment service. The evaluation asks: who uses it, whether time to senior assessment changes, whether admission rates and 7-day returns change, what patients and carers experience, and whether access differs by geography.

Data include monthly activity and outcomes for 12 months, travel-distance categories, brief patient interviews and staff workload estimates. The evaluation finds fewer overnight admissions but higher next-day community workload and lower access from remote areas.

The decision is therefore not “successful/unsuccessful”. The service continues, but transport pathways and community staffing are redesigned and equity metrics remain in the monitoring set.

Structure the report around the decision questions, methods, limitations, findings and implications. Separate observation from interpretation. Explain missing data, changes in service configuration and factors that limit attribution.

If the evaluation is later shared externally, be explicit that it was a local service evaluation unless it has been converted into or accompanied by research. Publication does not retrospectively transform a project into research, but plans for dissemination should be considered early because they may influence governance and consent requirements.

Include a clear recommendation: continue, adapt, scale, pause, stop, or gather further evidence - and state what evidence drove that recommendation.

For an existing service, write five evaluation questions. For each, specify the indicator, data source, method, frequency, population, potential bias and decision it will inform. Include at least one patient-experience measure, one equity measure and one unintended-consequence measure.

Then review whether the project still fits service evaluation. If the design deliberately changes care to test a hypothesis intended to generate generalisable knowledge, seek research advice. Use the Stage 4 Service Evaluation Matrix to document the plan.

• Calling work “service evaluation” simply to avoid research governance.
• Collecting many metrics without a decision question.
• Using only satisfied service users for qualitative feedback.
• Claiming causation from a simple before/after comparison.
• Ignoring workload shifted elsewhere in the system.

1. Healthcare Quality Improvement Partnership (HQIP): Guide to managing ethical issues in quality improvement or clinical audit projects, updated October 2025. https://www.hqip.org.uk/guidance/guide-to-managing-ethical-issues-in-quality-improvement-or-clinical-audit-projects/
2. Health Research Authority (HRA): UK Policy Framework for Health and Social Care Research, version 3.4, updated 28 April 2026. https://www.hra.nhs.uk/planning-and-improving-research/policies-standards-legislation/uk-policy-framework-health-social-care-research/uk-policy-framework-health-and-social-care-research/
3. HRA Decision Tools: Is my study research? and Defining Research terminology. https://www.hra-decisiontools.org.uk/research/

Educational reference. Verify current local governance requirements before starting a project.