Test Submission 15/12/22 22:30 Introduction The Welsh Ambulance Services NHS Trust (the Trust) serves over three million people across the whole of Wales. The Trust attends over 250,000 emergency calls a year, over 50,000 urgent calls and transports over 1.3 million non-emergency patients to over 200 treatment centres throughout Wales and England. The Trust is the host for the 111 service, a 24-hour health advice and information service for the public and the front-end call handling and clinical triage elements of the GP out of hours services. The Trust employs around 3,500 people, approximately 70% of whom are within the emergency medical services (EMS). The Trust operates from 90 ambulance stations, three contact centres, three regional offices and five vehicle workshops (Welsh Ambulance Services NHS Trust (WAST), 2022a) This report focusses on the EMS part of the Trust. EMS includes paramedics and emergency medical technicians who deliver emergency care to patients across Wales. In 2015 the Trust introduced a Clinical Response Model (WAST, 2020a) following endorsement by the Welsh Government of a change to a clinically focussed model of ambulance care in Wales. The model involves a move away from purely time-based performance measures for all but immediately life threatening (‘Red’) calls, and the introduction of clinical indicators (see Appendix A) to measure the quality of evidence-based patient care for certain conditions. It had previously been noted in England that there was a convincing case for ambulance services to develop clinical performance indicators to move from response time indicators due to changing clinical demands on services (Siriwardena et al, 2010) Each clinical indicator (CI) consists of a ‘care bundle’ of a number of individual evidence-based criteria that relate to the same clinical condition. Care bundles are a structured way of improving the process of care and patient outcomes. They comprise a small straightforward set of practices, which when performed collectively, reliably, and continuously have been shown to improve patient outcomes (WAST, 2022b). A 2012 paper reviewing international approaches to measuring quality in EMS systems noted that evidence-based care bundles can be good gauges of effectiveness relating to specific clinical conditions and patient outcomes (El Sayed, 2012). The Trust implemented an electronic patient clinical record (ePCR) system Wales-wide in April 2022 which has the capacity to record and store a large amount of clinical data. The Trust had previously used digital pen technology, introduced in 2015, to produce Patient Clinical Records and to generate data to report on clinical indicators. There has not been a facility for team leaders to generate reports on their own team performance on CI care bundles since the end of reporting with digital pen data in November 2021. Prior to the implementation of ePCR there was also a change in the arrangements for the clinical supervision of EMS clinicians. This responsibility moved from Clinical Team Leaders (CTLs) to a new role of Senior Paramedic (SP). The SP role includes responsibility for the clinical supervision of a defined group of EMS clinicians (Paramedics, Emergency Medical Technicians and Urgent Care Assistants) including feedback on clinical performance and compliance with clinical indicators and care bundles. It is not currently possible to use ePCR clinical data to monitor and provide feedback on standards of care by individual clinician or by a defined group of clinicians. It is proposed to develop a clinical indicator feedback system to enable reporting on CIs and other clinical performance data, to enable SPs to provide meaningful, detailed and timely feedback and clinical supervision. The system will help to support proactive, planned clinical supervision, which has recently been promoted by the Association of Ambulance Chief Executives (AACE, 2021). Justification The proposal is to improve the use of ePCR clinical data to enable meaningful, detailed, and timely feedback on clinical performance to clinicians. This will be achieved through: • The development of a new clinical indicator feedback system to enable Senior Paramedics to generate care bundle compliance reports on individuals for whom they are clinically responsible. • To link this system to a reporting function to enable SPs to generate information on other measures e.g. conveyance rates. The proposal is necessary and important as ePCR clinical data is not currently available for reporting by individual clinician or by Senior Paramedic team (the defined group of ambulance clinicians for whom SPs are clinically responsible). There is also currently no system to record and keep SP teams up to date. Therefore, SPs are not able to run reports on their own team’s clinical performance using ePCR data to provide detailed and meaningful individual and team feedback. Figure 1 shows the business problem and causes in the form of a simple fishbone diagram Figure 1: Simple root cause analysis of the business problem Recent clinical data assurance work on Trust-wide CI performance based on raw ePCR data has shown that there is a need to explore alternative (more accurate) approaches for monitoring individual standards of care. A recent qualitative research study (Wilson et al., 2022) concluded, in line with previous research in this area, that EMS professionals have a strong desire for feedback. Feedback on clinical performance or patient outcomes can change professional behaviour, improve clinical outcomes, and have a positive effect on staff mental health (Ivers, N. et al., 2012, and Michie S., Williams S., 2003 cited in Wilson et al., 2022) The system improvement initiative proposed here will provide a foundation for structured, relevant, and regular clinical feedback to EMS personnel, based on evidence-based clinical indicators. Clinical feedback forms part of effective clinical supervision (AACE, 2021), and therefore it is worth considering the reasons behind recent developments in promoting clinical supervision of ambulance clinicians. A key driver of the development of a clinical supervision framework for UK ambulance services is the aim of reducing variation in avoidable conveyance rates (NHS Improvement, 2018, ‘the Carter Review’). The AACE clinical supervision framework (2021) also recognises that a study (Van der Gaag et al, 2017) commissioned by the Health and Care Professions Council (HCPC) into reasons for the disproportionately high number of complaints to the HCPC highlighted that inadequate supervision was key factor associated with the findings. There is limited literature available relating to similar systems relating to that being proposed (see Literature Review). However, the AACE (2021), notes that in developing a clinical supervision model ambulance trusts must embrace digital technology wherever possible, to surmount logistical and geographical barriers. The system proposed is not an electronic performance monitoring (EPM) system, such as that used in call centres to enable supervisors to track an employee’s workflow in real time (Association for Psychological Science, 2014). It is a clinical system using ePCR data to give an indication of compliance with evidence-based care bundles. The data is to be used in a supportive and balanced way, in the context of meaningful clinical supervision. Aim & Objectives Aim To implement a clinical indicator feedback system to enable monitoring and feedback to clinicians on care bundle compliance using ePCR data. Objectives • To develop, test and implement by the end of June 2023 a clinical indicator feedback system (CIFS). • Alongside the development of the CIFS to ensure that responsibilities for clinical feedback, performance measurement and improvement are aligned with the Trust’s operational, clinical, governance, quality, and assurance arrangements. • To develop and roll out training on the purpose and use of the CIFS to all EMS, Senior Paramedics, Health Board Clinical Leads, and other affected staff in by end of 2023. • To fully embed the CIFS and the associated reporting function in the personal appraisal development review (PADR) process and the Trust’s Operational Performance Framework by the end of 2023/24. Literature Review The literature review search strategy was based on locating literature which focussed on feedback to ambulance clinicians: • on clinical performance, and/or • in relation to clinical indicators A range of searches were carried out using different key words in Google Scholar. The initial search terms and results are shown in Table 1. Table 1: Literature review search results Date Search terms No. of papers identified 22/11/22a Advanced search, in title (ambulance OR emergency or EMS or prehospital) (feedback OR supervision OR outcome or clinical) (system OR systems OR digital) 437 22/11/22b Advanced search, in title (ambulance OR emergency or EMS or prehospital) (feedback OR clinical OR indicators) (system OR systems OR digital -department -departments -decision 224 22/11/22c Advanced search, in title, date range 2000-2022 (ambulance OR EMS OR prehospital) (feedback OR clinical OR indicators) (system OR systems OR digital) -department -departments -decision 36 9/12/22 Advanced search, in title, 2000-2022 Ambulance feedback 19 The articles found in the initial searches (22/11/22a 22/11/22b) were not reviewed due to high number of results. The articles found in search 22/11/22c were excluded as they did not deal with feedback to ambulance clinicians on clinical indicators, and in one case was not dated. These searches did not produce any examples of a clinical indicator feedback system for ambulance clinicians. However, the results from search 9/12/22 produced some papers relevant to the issue of feedback on paramedic performance. Of the 19 results, 13 were excluded as the feedback related to non-clinical performance eg driving standards; feedback on performance on manikins; feedback related to dispatch rather than direct, face-to-face patient care; duplicated results. The remaining six papers are summarised in Table 2. Table 2: Literature on feedback to ambulance clinicians (by year of publication) Author (Year) and location Title Main findings and conclusions Jenkinson, E., Hayman, T. and Bleetman, A., (2009) UK Clinical feedback to ambulance crews: supporting professional development There was a positive response from paramedics following the introduction of a formal feedback service for ambulance crews on patient outcomes following hospital admission. Ambulance service personnel are interested in the in-hospital care and outcome of their patients, following their early diagnosis and management. Brandling et al (2019) UK Investigating the use and effect of a patient outcome feedback intervention (POFI) on ambulance staff. (Abstract) A digital patient outcome feedback intervention (POFI) was designed and tested to report patient outcomes after hospital conveyance. Feedback is desirable, but requires a commitment by ambulance services and hospital sites to share data. This project is the only one known using a digital mechanism. Respondents felt feedback is likely to improve decision-making confidence, enhance self-efficacy and patient safety, and contribute to learning through self-reflection. Eaton-Williams, P., Mold, F. and Magnusson, C., (2020) Studies from Germany, US, Netherlands, UK, Norway, Sweden, Canada Effective clinical feedback provision to ambulance clinicians: a literature review. Feedback mechanisms for ambulance practitioners have often been shown to improve clinical performance, although demonstrating subsequent benefits to patient outcomes has been more difficult. The initiation and maintenance of a feedback system require significant resources for both retrieval and delivery of information. As well as consent and information governance issues, this may account for the scarcity of such mechanisms. A number of factors should be considered for effective feedback delivery eg when and by whom feedback is delivered; feedback should be balanced, not punitive. Further research is required to evaluate the effects of clinical feedback to ambulance clinicians on their clinical development, their wellbeing, and their patients’ outcomes. Wilson et al (2021a) International The types and effects of feedback received by emergency ambulance staff: protocol for a systematic mixed studies review with narrative synthesis. The review will be useful to guide the development of future prehospital feedback programmes and interventions, for which there is growing interest. An evidence base for the design of prehospital feedback is important for the enabling effects of both developing electronic health record systems and the accompanying capability for data linkage across organisational and care system boundaries (Porter et al., 2020). Such developments in healthcare informatics may afford important opportunities to develop more effective feedback for prehospital practitioners in the near future. Wilson et al (Abstract) (2021b) International The types and effects of feedback received by emergency ambulance staff: a systematic mixed studies review with narrative synthesis Narrative synthesis revealed a wide range of prehospital feedback types with the majority of studies focusing on performance or patient outcome feedback. The effects of feedback encompassed both personal and professional development, such as improved decision-making, reflection and closure. Mechanisms of providing prehospital feedback varied from electronic dashboards to case reviews and performance appraisals. Further empirical research is required to explore whether the published literature reflects current prehospital practice, as well as to determine the when, what, how and why of ambulance staff desiring feedback Wilson et al (2022) UK The role of feedback in emergency ambulance services: a qualitative interview study. Types of feedback include patient outcome feedback and performance feedback. In line with previous research, the study highlights EMS professionals’ strong desire for feedback. The study suggests a typology of prehospital feedback and presenting a unique insight into the motives for feedback seeing using psychological theory. A logic model for prehospital feedback interventions was developed to inform future research and development int pre-hospital feedback. The literature reviewed (e.g., Brandling et al, 2019, Wilson et al, 2021b) reveals that formal feedback to ambulance clinicians usually takes on one or both forms: • Performance feedback i.e., on clinical care provided in relation to specific conditions eg evidence-based clinical indicator care bundles • Patient outcome feedback i.e., confirmed diagnosis, treatment following hospital admission The literature indicates the benefits of feedback, and the potential for using clinical data to support other forms of feedback eg direct supervision (‘ride-outs’). This data could take the form of care bundle compliance (from ambulance ePCR data only) and patient outcomes (this would require linkage of ambulance data to hospital data which is not a reality yet in Wales). The majority of literature reviewed does not mention any form of digital information system to facilitate clinical feedback. Where a digital mechanism was used (Brandling et al, 2019) there is no mention of the project management methodology used. Methods Justification for the selected project management methodology PRINCE2 (Projects in Controlled Environments) will be used as the project management methodology. PRINCE2 is a tried and tested generic methodology, widely used across the public and private sectors for projects of any size (Axelos, 2017). PRINCE2 is already used within the Welsh Ambulance Services NHS Trust eg a tailored PRINCE2 approach was used in the development of the ePCR (WAST, 2021). PRINCE2 will provide a clear and structured approach for ensuring the successful delivery of the project. Description of PRINCE2 The PRINCE2 methodology is based on 7 principles. These are listed below, together with how they relate to the proposed project: 1. Continued business justification – the project is required as a key requirement to support effective clinical supervision, which in turn is a key driver of patient care and staff wellbeing. 2. Learn from experience – the project will include lessons learnt from previous information systems projects carried out within WAST. 3. Defined roles and responsibilities – everyone who is part of the project will be clear on how they are contributing to it’s successful delivery. 4. Manage by stages – each project stage will planned, monitored and managed before moving to the next stage. 5. Manage by exception – it will be clear who has authority to make decisions in relation to key tolerances (eg cost and time) at each stage 6. Focus on products – the product to be delivered (the Clinical Indicator Feedback System) and the expected benefits (eg reporting functionality - what reports can be generated) will be clearly defined at the project initiation stage to enable a clear measure of success. 7. Tailor to suit the environment – the PRINCE2 approach will be tailored to suit the proposed project. This will ensure clarity in all aspects of the project, while ensuring the project management process is no more bureaucratic than is necessary to achieve such clarity. Implementation of PRINCE2 methodology The project will be managed using key elements tailored from the PRINCE2 methodology (Axelos, 2017): Initiation stage In this stage the project board will be convened, and the project manager appointed. The project board will include representatives of the executive, suppliers (system developers) and users (clinical and operational staff). Project Initiation Document (PID) to be developed by the project manager and agreed with the project board. This will clearly define the scope of the project and be used to measure the success of the projects. The PID will include the following sections: • project definition • project approach • business case • project management team structure and role descriptions • quality management approach • risk management approach • change control approach • communication management approach • project plan • project controls • tailoring of PRINCE2. Delivery stage In this stage the new system will be designed, installed, and tested prior to implementation. The project manager will provide regular reports to the project board on progress and any issues arising. Closing the project This process provides a fixed point at which acceptance of the new system is confirmed and the project’s objectives have been achieved. The project manager will use the process activities to prepare to meet with the project board to request authorisation to close the project. Objectives of the process: • to gain user and operational acceptance for the product, • to review the project against its baseline, • to assess any benefits that have already been realized, • to ensure post-project benefit reviews are planned, • to ensure that provision has been made to address all open issues and risks with follow-on action recommendations. Project Plan Figure 2 shows the project plan in the form of a Gantt chart, over 25 weeks. Figure 2: Project plan – Gantt Chart Issues and caveats There are known issues that require clarity during the planning and implementation of the project: • Clear definition of incidents attended by geographical areas eg by incident postcode – to enable reporting by locality, Health Board and regions. • Allocation of incidents (and therefore ePCRs) to individual clinicians. All staff who attend at incident is allocated that record, regardless of whether they have directly contributed to patient care. This must be considered when providing feedback to individual staff especially if the case is non-compliant with the care bundle and the staff member receiving feedback had no or limited clinical input into the patient care episode. • Clinical indicator data and feedback must include the caveat that justified exceptions (reasons for not providing a specific element of the care bundle) are not always identified in automated data, for example patient refused analgesia documented in the narrative free-text fields, which is not detected by automated reporting. There may be scope to improve the accuracy of recording data by clinicians if a data field (eg button selected for pre-defined exceptions) could have been used rather than a free-text text. Improved accuracy will help contribute to the overall quality of the electronic patient record (American Health Information Management Association, 2012, cited in Coiera (2015)). • Whilst digital technology will support and facilitate clinical feedback using meaningful data the feedback should be delivered in the context of a human relationship of trust (Morrison et al, 2017; AACE, 2021). • • Individual patient records retrieved for feedback and discussion with clinicians will be appropriately redacted in line with Caldicott guidelines and GDPR REF? Human factors Morrison et al (2017) interviewed paramedics and identified some important issues relating clinical performance feedback. Feedback was perceived positively, however, the weaknesses of feedback systems were also mentioned, together with perceived barriers to obtaining meaningful feedback. Feedback and its relation to mental health was a theme, with a potential for both positive and negative effects. Other key findings: • Performance feedback on pre-hospital care and outcomes is generally infrequent and inconsistently delivered. • Research has produced unreliable results regarding the value of feedback to health professionals—potentially reflecting the variation in content of feedback, the context of the feedback, and the mode of delivery. • Not all feedback systems achieve their intended aim of performance improvement. Some may not provide enough meaningful information to individuals so they may actually change their practice. • Health professions education literature has focussed on the social phenomenon of feedback between teachers and learners. • In quality improvement literature, there is a much discussion on data-driven feedback and audit systems where practicing individuals are simply provided information. It is important that the system being proposed takes account of these factors, particularly in terms of the context of feedback delivery and training for clinical supervisors eg Senior Paramedics who have the responsibility for the clinical leadership of a defined group of ambulance clinicians, who in turn are directly providing face-to-face patient care on a daily basis. Evaluating the impact of the project FITT / SUS? An evidence-based health informatics framework will be used to evaluate the system adoption eg the Clinical Adoption Framework (CAF), as described by Craven et al, 2016. This will include consideration of people and organisational factors, system and information quality and user satisfaction. The project is part of the wider implementation of clinical supervision in the Trust. AACE (2021) describes some of the measurable outcomes of effective clinical supervision which will be used to evaluate the impact of project following implementation: • Increased clinical indicator care bundle compliance • Increased job satisfaction and staff retention • Reduced workplace stress • Reduced staff burnout • Reduced staff turnover • Reduction in avoidable conveyances. CS – Task Oriented Content (Pearce et al 2013, cited in AACE, 2021) It is worth noting that in terms of clinical supervision, there is a lack of clear guidance on how to measure the performance of an effective clinical supervision model (AACE, 2021). However, the Trust will review effectiveness of clinical supervision through the Quality Committee, and monitoring at Board level. Part of clinical governance (Francis report – just….. cited in AACE, 2021) Discussion Discussion placeholder – influence wider system improvements Relate this to other relevant domain (digital tech in h&c) knowledge from the literature… Discuss possible extension or downstream benefits of your intervention…. It is anticipated that following the project implementation that individual clinicians will be able to search their own clinical performance e.g clinical indicator care bundle compliance. Feedback reporting to be allied with planned data linkage from the Trust ePCR data to the Welsh Clinical Portal (WCP) to enable feedback to clinicians on patient outcomes. Part of the Trust’s Digital Strategy (Welsh Ambulance Services NHS Trust, 2020b) is to report on patient outcomes through successful implementation of ePCR with other national systems. The Trust Digital Strategy includes the vision ‘to provide the best data, at the best time, presented in the best manner to drive the best decision’ (WAST, 2020b, p19). The Clinical Indicator Feedback System will provide a foundation for the development of further feedback options to clinical staff on patient outcomes following admission to hospital, e.g., on confirmed diagnoses, further treatment, and outcome at discharge. The extension and downstream benefits of the current initiative will include linking in with work already underway in the Trust with the National Data Resource (NDR) programme to pilot new ways of handling, storing and securely sharing data across NHS Wales in close to real time. The current initiative goes hand in hand with looking at the architecture of Trust data, together with international standards, such as SNOMED-CT to make it more accessible and understandable, to drive meaningful decisions as soon as it is available (WAST, 2020b). AACE, 2021 p26 – Ambulance Dataset (ADS)…… It is anticipated that in line with the AACE (2021) that the CIFS will, as part of clinical supervision framework, will support the development of a set of evidence based clinical protocols which provide staff with the confidence for safe decision making resulting in reductions in avoidable conveyances and effective patient care. The increase of safe discharge patients at home where appropriate will contribute to the Trust’s strategic aim of ‘inverting the triangle’ (Welsh Ambulance Services NHS Trust, 2022c). This is a way of expressing the Trust’s ambition to ensure that patients receive the right advice and care, in the right place, every time (See Appendix B). Future downstream benefits, clinical indicator data will link with other Trust data to WCP Conclusion Recently published studies on feedback to emergency ambulance staff (eg Wilson et al, 2021b, and Wilson et al, 2022) indicate that the effective use of clinical performance and outcome data as a part of providing meaningful feedback to ambulance clinicians is relevant and growing issue. As the NHS continues to adapt and respond to external pressures and changes the intelligent use of clinical data will be a vital part of the twin aim of supporting staff to be their best and to drive the best possible evidence-based patient care. The linkage of ambulance and hospital data, enabling feedback to clinical staff on patient outcomes is likely to be the next project undertaken following the adoption and implementation of the Clinical Indicator Feedback System.