Evaluation hero

Service Evaluation

A service evaluation will seek to answer, “What standard does this service achieve?”. It won’t reference a predetermined standard (that would be ‘audit’ – see the Clinical Audit Department) and will not change the care a patient receives (that would be ‘research’).

Where an evaluation involves patients and staff, it is best practice to seek their permission, through consent, for their participation. The degree to which consent is taken can vary from asking someone if it is OK to seek their feedback on the service, through to a formal process using consent and participant information sheets.

Applicants wanting to do a service evaluation as a student project should apply using the SE1(S) form. Applicants wanting to do a service evaluation as a general, non-student project should apply using the SE1(G) form.

Please allow enough time for your application to be reviewed. While we will endeavour to review your application as soon as practicably possible, our small team are currently experiencing high workload pressures. Applicants must not start their service evaluation until they receive written approval from the R&E Department.

What is a service evaluation?

Many people make claims about their services without sound evidence to inform their judgements. A well planned and executed service evaluation will provide some of the following:

  • Evidence to demonstrate value for money.
  • A baseline from which to measure change.
  • Evidence to demonstrate effectiveness.
  • Evidence to demonstrate efficiency.
  • Evidence to demonstrate benefits and added value.

Service evaluations don't require NHS Research Ethics review but do need to be registered with, and approved by, the Research and Evidence Department before they can commence. This will include checks that the project is feasible (e.g., the service has capacity, appropriate timeframes) and carried out in line with Trust standards. Registration ensures the R&E Department know what type of activity (research and service evaluations) is happening in which services.

Please note, the Trust primarily processes data to deliver healthcare. The data cannot be used for other purposes unless the law allows ‘secondary processing’. Research and service evaluation are both considered secondary processing.

Data collected through service evaluations must be obtained, recorded and stored as defined by all relevant data standards, including The General Data Protection Regulation (GDPR) and the Data Protection Act 2018. Your evaluation must be designed in such a way that privacy is built into all aspects that involve data.

A great source of information is also available from the Evaluation Works.

What do I need to do to undertake a service evaluation?

Be clear about what you want to evaluate.

This will shape how you conduct the evaluation and define what information you will need to collect and where the data is.

 

Identify all stakeholders

Stakeholders in the success of your evaluation may include, amongst others service managers, commissioners, staff and patients. Be sure to engage them as early as possible to ensure they understand your work. Involving them early will help you shape your application to ensure it is successfully delivered.

 

Plan your project

Planning is paramount and you should allow enough time to produce the necessary paperwork and seek approval. You must also consider how and where data will be recorded and stored. Wherever possible you should use non-identifiable data. Data must be anonymised or pseudonymised and kept secure within the Trust. The Information Commissioner's Office define anonymised data as “…data that does not itself identify any individual and that is unlikely to allow any individual to be identified through its combination with other data.” Unless you have obtained prior agreement from the Research & Evidence Department, only fully anonymised data may be taken outside the Trust (e.g. to be stored on university servers).

All project members – Trust employees and those on honorary contracts – requiring access to data or Trust systems should have completed the NHS England Information Governance (IG) eLearning module. A paper version of the training can be provided where members do not have electronic access. Project members receiving anonymised data only are not required to complete the eLearning module.

 

Develop your paperwork

The SE1(G) or SE1(S) forms set out the information you need to provide. This covers the aims, objectives (what you will do to meet the aims), methodology (including data handling), analysis, and dissemination. Please follow the additional guidance on the form so that the form is completed in full.

To make it easier for our Governance Facilitators to review, students should complete the SE1(S) form in full rather than refer to a protocol that their university may have required them to complete.

Depending on how you plan to gather the data for your evaluation, you may need a participant information sheet and a consent form. You should seek appropriate informed consent from participants where necessary. Consent should be explicit verbal or written. Written consent should be sought where participants are identifiable or where their identifiable data is involved, or qualitative methods are being used. The Research and Evidence (R&E) department can help you determine what you will need; templates are available.

All forms must be version controlled to ensure approved ones are being used.

Your project must be approved by, and registered with, the R&E department prior to starting.

Prior to submission, applicants are encouraged to discuss their proposed project at one of the Trust’s Research Clinics. These are held via MS Teams and can help to improve the application. Please contact research@nottshc.nhs.uk to book on to a Research Clinic.

What about consent and accessing data?

Consent

Best Practice in the Ethics and Governance of Service Evaluation guidelines in the NHS Evaluation Toolkit provides the following guidance. Where service evaluations involve human participants, this should include seeking appropriate informed consent, taking into account risk, vulnerability and capacity of the participant. For example:

  • Consent should be appropriate to the participant's age, learning ability, language, religious and cultural beliefs.
  • Consent should be explicit verbal or written consent where participants are identifiable or where their identifiable data is involved (or qualitative methods are being used).
  • Consent may also be implied through the completion of activities such as surveys.
  • All information provided to potential participants about the evaluation should be accessible to enable fair and equitable access to the study. It should take account of the potential sensitivities, emotional impact and distress that may arise from the service evaluation.
  • All information should make it clear that participation is voluntary, and they can withdraw at any time without adverse consequences.

 

Accessing data

You may want to access patient data to help you complete your service evaluation. Where possible this should be anonymised data e.g. from the Applied Information Department.

Service evaluations are considered secondary use of data. This means that there will need to be legal bases under the UK GDPR and the DPA 2018 to allow you to process any pseudonymised or fully identifiable data. The Common Law Duty of Confidentiality must also be satisfied.

To ensure that the Common Law Duty of Confidentiality isn’t breached, you will need a legitimate right to be able to access identifiable confidential data to support your evaluation for example, data within Electronic Health Records. You may be able to access the identifiable confidential data that you need for your evaluation if you are or were part of the patients’ direct care team. Otherwise, someone else with legitimate access should access the data and anonymise it on your behalf.

Alternatively, if the benefits that will be achieved by completing the evaluation outweigh the Trust’s obligation to maintain confidentiality and if any potential risks to the patients’ resulting from a breach in their confidentiality are low, access to identifiable confidential data may be approved by the Data Protection Officer or Caldicott Guardian on the grounds that it would be in the ‘public interest’ to provide access.

You should also remove any identifiers to anonymise the data at the earliest opportunity.

Apply to register your project with the Research and Evidence department

The completed SE1(G) or SE1(S) form and any additional documents (e.g., copies of data collection forms and any interview topic guides) should be sent to research@nottshc.nhs.uk. Incomplete forms will be returned to the applicant to request completion. The review will not start until we have received a completed form and all relevant accompanying documents. Please contact the R&E Department if you need advice.

Following an initial review from the R&E Governance team, you may be invited to discuss your application in a MS Teams call with the Governance Facilitator. This allows applications to be processed more efficiently as it gives the applicant the opportunity to address any queries or concerns; queries will need to be resolved before approval. Applications are usually approved faster when the applicant has attended a research clinic before they submit. For non-Trust staff, a research passport / Letter of Access may be required where the applicant does not hold a contract with the Trust.

The R&E team may forward to the Population Health Unit, any non-student application the applicant identifies as potentially having an impact on service configuration/development, a financial impact or are part of the Trust's Integrated Improvement Plan. This will be to see if the application might benefit from input from the Population Health Unit. 

Applicants must not start their service evaluation until they receive written approval from the R&E Department.

Please allow enough time for your application to be reviewed. While we will endeavour to review your application as soon as practicably possible, our small team are currently experiencing high workload pressures.

Conducting and writing up your project

Conducting your project

With good planning and engaged stakeholders your project should run according to plan. Where we see problems arise it is usually because of something that was not considered, planning was not thorough enough, or staff not having the time to do the evaluation.

Where amendments to the evaluation are necessary (e.g., timeframes, changes to the study team), the applicant should discuss and agree these with the R&E Governance Team beforehand. This should prevent deviations to the approved study.

 

Writing up your project

It’s important that your project is written up at the end to ensure what was learnt can be shared and used to make improvements. As a minimum, a Final Report (FR1) should be completed to provide a summary of the evaluation – see template in 'Forms and templates for service evaluations'. Students will often need to do a larger report as part of their course or qualification. Students should still complete the FR1 as these are more likely to be read by the service than a full dissertation.

A copy of the FR1 will be put on our intranet (Connect) page – reports should be sent to research@nottshc.nhs.uk.

In addition to providing the final report to R&E and the service, the study team should present their findings to the service or other relevant Trust meetings if requested.

We also want to capture the impact of your service evaluation so please give some thought as to how this can be achieved.

Forms and templates for service evaluations

 

 

Rate this page or report a problem

Rate this page or report a problem
Rating
*