The high priority that service users and carers attach to crisis care and their reservations regarding CRT implementation were the main drivers for the CORE programme. Public involvement was an integral part of planning and conducting the research throughout the CORE project. In this section, the structures through which patient and public involvement (PPI) was facilitated during the CORE study are summarised, including the types of PPI activity undertaken. The impacts of public involvement on the study are briefly discussed in Public involvement activities. In addition, interviews with members of the CORE study service user and carer working groups on the experience and impact of PPI in the study were conducted by an independent researcher. A report summarising feedback from these interviews is also provided at the end of the synopsis (see Impact of public involvement on the study).
A budget for public involvement in the study was included in the grant awarded by the funders. The study public involvement co-ordinator post was funded as a NHS band 5 role. Working group members were offered paid involvement at a rate of £15 per hour (time to review study documents prior to meetings was included in payments). Working group members’ travel costs to meetings were also reimbursed. The service users in the research group (led by Thomas Kabir) from the now-disbanded MHRN were a valuable source of advice with PPI arrangements on the study, regarding recruitment and payment arrangements and facilitating access to free benefits advice for working group members. The MHRN also helped set up two consultation meetings with service users and carers during preparation of the research grant, allowing service user and carer involvement with initial study planning. The CRT model has not been highly specified in the literature, as its critical ingredients are not yet established.6 There is limited evidence about how best to implement CRTs and what type of interventions to deliver within them.4 Workstream 1 developed the CRT model by investigating what constitutes best practice in CRTs and how difficulties in CRT model implementation identified in recent reports may be addressed. The approach that was adopted in workstream 1 was based on the US EBP programme.36 Central to the EBP method is the development of a fidelity scale, based, when available, on empirical evidence on components of the model associated with clinical effectiveness, and on stakeholder qualitative evidence and expert consultation when this is unavailable. Development of fidelity measures for complex interventions in mental health has been advocated not only as a means to define an intervention and measure services’ adherence to the model specified, but also to support service improvement.56 Following fidelity scale development, a service improvement programme including manuals, training materials and guidance on implementation was designed to help teams achieve high fidelity and hence best practice. The first part (module 1) of workstream 1 focused on investigating and defining best practice in CRTs. First, we reviewed current evidence regarding critical ingredients of CRT care and of relevant guidance. Second, we conducted a national survey of CRT managers in order to examine current service organisation and delivery and any initiatives to improve CRT practice. Third, we explored the perspective of service users, carers, professionals and experts on best CRT practice and their suggestions for service improvement. Findings from these studies were used to inform the emerging development of a model of CRT practice. In module 2,we aimed to develop and test a CRT fidelity scale, designed to assess how far best practice as defined following module 1 is achieved. The fidelity scale was developed through a rigorous and systematic process, using stakeholders’ views to prioritise potential fidelity items drawn from the available evidence. The CRT fidelity scale was subsequently tested in a national survey of CRT model fidelity. We then developed a service improvement programme (SIP) for CRTs, with the fidelity programme as the central framework. The CORE SIP was modelled on the EBP project’s implementation resource kits,57 with content derived from the fidelity scale and provision of good-practice examples from the national survey. The final stage in workstream 1 (module 3) involved a mixed-methods assessment of the impact of the SIP for achieving high fidelity to the CORE-defined model of good practice. Methods for this evaluation were finalised in the previous modules to ensure that it corresponded to key goals embodied in the CRT fidelity scale and SIP. Twenty-five CRTs were recruited to participate in a cluster randomised trial, with teams as the unit of randomisation: 15 of these were randomly allocated to the experimental intervention and 10 were randomly allocated to the control intervention. The aims were to establish (1) whether or not fidelity scores increase following the SIP, (2) whether or not teams receiving the SIP (the experimental intervention) achieve better service user experiences and better performance on key indicators of service use and (3) whether or not higher fidelity scores are associated with better outcomes at a team level. In addition, qualitative methods were used to explore participant experiences of the SIP. See Appendix 1 for the published report of this work.58 The report is accessible at the following URL: https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-015-0441-x (accessed 28 March 2019). The review is registered with PROSPERO as CRD42013006415. The aim of this review was to systematically acquire and synthesise evidence regarding critical components and key organisational principles of CRT services. In the review, quantitative evidence on associations between specific CRT characteristics and outcomes was prioritised. However, initial searches confirmed the belief that clear empirical evidence was likely to be limited. Literature on the views of stakeholders, including service users, carers, clinicians and service leaders, about best practice in CRTs was reviewed. In addition, the recommendations that government and expert organisations make about CRT service delivery and organisation were examined. A systematic review was conducted. The databases MEDLINE, EMBASE, PsycINFO, Cumulative Index to Nursing and Allied Health Literature (CINAHL) and Web of Science were searched to November 2013, without time limit or geographical restrictions. Studies were included from anywhere in the world and in any language. A further web-based search was conducted for published government and expert guidelines on CRT services; this was restricted to England. Full search terms are provided in the published report.58 The following types of study were included:
A narrative synthesis was conducted, based on guidance from the Centre for Reviews and Dissemination.59 Each of the above types of study was separately investigated. Statistical meta-analysis was not feasible because of the heterogeneity of included studies with regard to study aims and design, outcomes assessed, service settings and characteristics. The mixed-methods appraisal tool (MMAT),60 a tool that allows quality rating of a variety of types of study, was used to assess the quality of individual studies. Sixty-nine studies and documents were included in the review. The composition and activities of CRTs varied substantially across studies. Studies also varied in quality, with the total mean score on the MMAT scale60 indicating moderately high quality. Overall, quantitative comparison studies and CRT surveys provide relatively little evidence regarding the specific characteristics of CRTs that are associated with positive outcomes. Many studies of CRT outcomes give relatively little detail regarding the characteristics of the CRTs, making it very difficult to draw conclusions about associations between service elements and outcomes. One comparison of two CRT models found the presence of a psychiatrist within the team to be associated with reduced admissions, and a national survey suggested an association between 24-hour opening and lower admissions. Little other relevant evidence was obtained. Qualitative studies and government and expert guidelines provide more detailed specification of best practice in CRTs. Stakeholders emphasised accessibility, integration with other local mental health services, provision of time to talk and practical help, treatment at home when possible and continuity of care. CRT guidelines prioritised the provision of a 24-hour, 7 days per week multidisciplinary service, including a psychiatrist and medical prescriber, relapse prevention planning on discharge from the CRT and a gatekeeping role for CRTs in controlling inpatient admissions. The need for adequate staffing to meet service demands and high quality of staff training were also frequently emphasised. In order to establish the critical ingredients of CRTs, this review used a systematic search strategy to identify all available types of evidence. The multidatabase search for relevant research studies was supplemented with hand-searching of reference lists and contacting authors about conference abstracts. Although this search included international literature regarding CRT services and closely related models, the search for government and expert guidelines was confined to England because of resource limitations and challenges in identifying and accessing such sources. Another limitation was the fact that because of the wide variation among studies and incomplete reporting of CRT characteristics, it was not feasible to conduct quantitative synthesis of results or compare the clinical effectiveness of CRTs across studies. Furthermore, the MMAT60 is a relatively unsophisticated means of assessing quality, even though it is probably the best available single measure for synthesising the quality of studies using a mixture of methods. The tool treats different methodologies as equivalent and merges different components of quality into a single score, which is potentially reductive and misleading.61 The review did not exclude any papers on the basis of quality, as it was wished to fully assess the current evidence base regarding CRT services. As a result, some studies with lower quality scores were included, which may potentially compromise the strength of the conclusions. However, steps were taken to take account of the variability in the quality of included studies in our synthesis of the evidence. This extent to which empirically based critical ingredients of CRT good practice can be identified is constrained by the fact that only a very small number of studies address this question. Interpretation and synthesis of the studies that have been conducted are impeded by wide variations in models and outcomes investigated and lack of detailed information on service characteristics. Future studies of CRTs should provide a more comprehensive description of the CRT and comparison services, as recommended by the Consolidated Standards of Reporting Trials (CONSORT) guidelines.62 Although there was relatively little quantitative evidence regarding the critical ingredients associated with good CRT outcomes, there was a high degree of consensus from stakeholders and policy guidance about an optimal CRT model, broadly corroborating initial government guidance accompanying the national mandate to introduce CRTs.3 This provides a basis for development of a more highly specified CRT model and of a means to assess adherence to this model: a priority for CRT research. It would now be valuable to conduct further research into the characteristics of CRTs associated with good outcomes and good patient experiences. See Appendix 2 for the published report of this work.63 The report is accessible at the following URL: http://onlinelibrary.wiley.com/doi/10.1111/inm.12311/abstract (accessed 28 March 2019). A national survey of CRT managers was conducted in 2011/12. The aims of the survey were to map the provision of all CRTs in England and describe CRT organisation and service delivery, explore the teams’ adherence to key policy recommendations3 and compare CRT managers’ views on actual and desirable CRT characteristics. Between November 2011 and July 2012, the manager of every crisis resolution/home treatment team (or equivalent) in England was invited to complete an electronic survey. If a team manager could not be reached or wished to delegate the survey, another senior member of the team was invited to participate. The survey consisted of a 90-item questionnaire, which was based on previous national surveys of crisis services24,64 and was further refined after a pilot in four CRTs. The questionnaire covered a range of aspects of CRT organisation and service delivery, including exploring which interventions are available from CRTs. In addition, reports were elicited on local initiatives and priorities for service improvement, and views regarding ideal characteristics of CRTs. The survey could be completed online or as a telephone interview with a researcher. Data from the questionnaire were entered directly into a secure online system and were downloaded into a Microsoft Excel® (Microsoft Corporation, Redmond, WA, USA) spreadsheet and transferred to IBM SPSS Statistics (IBM Corporation, Armonk, NY, USA) for analysis. There were three stages of data analysis:
Through service mapping, we identified a total of 218 CRTs, with some CRT provision in all 65 mental health NHS trusts in England. One hundred and ninety-two CRTs (88%) took part in the survey and 184 respondents (84%) completed at least two-thirds of the questionnaire. Table 1 shows the composition of CRTs: nurses and psychiatrists were employed in almost all teams, but representation of other professions varied greatly from team to team. The survey also enquired about delivered interventions (Table 2). These were most often focused on prescription and delivery of medication, rather than on psychological, social or practical interventions. Only one-third of CRTs (62/187, 33%) reported always acting as gatekeepers to inpatient beds, as stipulated in the original policy implementation guidance.3 An even smaller proportion of CRTs (35/187, 19%) reported always attending formal assessments for compulsary admission under the Mental Health Act.29 Most CRTs (186/188, 99%) were also engaged in early discharge work, identifying inpatients who could go home earlier with support from the CRT. Team managers were asked to rate whether or not a number of aspects of service delivery were important, and to report whether or not they were actually able to provide these at present. Table 3 shows the results and indicates many discrepancies between actual and desired provision, for example in being able to provide practical support of various types to service users and in being able to spend extended periods of time with them. Thus, from the perspective of our work, the important overarching theme is that, although there are many gaps in areas such as psychological and social interventions and capacity to spend extended time with service users, such gaps do not result from these aspects of provision being seen as low priority. Rather, it appears that managers who were surveyed would have liked to deliver a much richer service than was possible in practice. This survey had a high response rate (88%), which gives confidence that the sample of respondents is representative of senior CRT staff in England. However, there are two main limitations. First, the use of self-reporting may have resulted in an over-reporting of CRT implementation because of social desirability bias. Second, the survey was cross-sectional in nature, thus shedding no light on changes over time. The survey suggested that the original policy mandate and guidance were not sufficient to achieve nationwide CRT implementation as intended. This supports conclusions from the US EBP programme36 that high fidelity to good practice is unlikely to be achieved without regular structured assessment of implementation. A need was thus identified for future research to develop and test tools to support implementation and monitoring of CRTs. CRT stakeholders’ views and priorities, including those of service users and carers, should also be investigated and taken into account in developing an empirically based optimum model of CRT care. Tracking service delivery nationwide by repeated surveys over time will also be important to achieving and sustaining implementation of a CRT model matching perceived needs. See Appendix 3 for a link to the full report of this work.65 The report is also accessible at https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-017-1421-0/open-peer-review (accessed 28 March 2019). Previous research has shown that there is insufficient empirical evidence regarding the critical ingredients of CRTs (see Module 1.1) and that CRTs are not fully implemented as intended (see Module 1.2). The EBP programme36 recommends drawing on a broad range of stakeholder views in which empirical evidence linking service content to outcome is limited. Therefore, in this qualitative study, the aim was to examine stakeholders’ experiences of CRTs and their views regarding best practice. Semistructured interviews and focus groups were conducted with different groups of CRT stakeholders: service users (n = 41 individual interviews), carers (n = 20 interviews) and practitioners (n = 147, comprising CRT staff, managers and referrers; nine individual interviews and 26 focus groups were conducted). The referrers group included clinicians from community mental health services, inpatient wards, liaison psychiatry services and general practitioners, thus representing a range of common referrers to CRT services. Participants were drawn from 10 mental health catchment areas across England. In addition, key experts (n = 11), involved in the development of CRTs internationally, were interviewed in order to provide a broad perspective on the history and theoretical origins of CRTs and on their intended aims and characteristics. Topic guides were developed in collaboration with project advisory groups of service users, carers and clinicians, representing CRT stakeholders, to make sure that priority issues and the concerns of each group regarding CRT practice were covered. The overall aim was to explore views from all perspectives about what constitutes best practice in CRTs. Aspects of CRT care identified by previous research as problematic were included. Interviews with service users and carers were conducted by trained peer researchers (service users and/or carers) whenever available; the rest were conducted by non-peer study researchers. Data from practitioners were collected via focus groups, when feasible, or via individual interviews if it proved too difficult to convene focus groups. All interviews were audio-recorded and transcribed. Data were analysed using thematic analysis66 supported by NVivo 9 software (QSR International, Warrington, UK). Using this analytical strategy, aspects of CRT work were explored that were perceived to be important, successful and unsuccessful, using both inductive and deductive approaches. Owing to the large corpus of data, a staged approach was used. An initial basic set of themes was developed, based on a small subsample of transcripts, in order to capture broad areas and organise the data. This set of themes was progressively elaborated through group discussions and an iterative process of reading and coding further transcripts. Later stages of analysis were focused on developing a nuanced understanding of parts of the data corpus relevant to our aims. Peer researchers, as well as members of project advisory groups representing all stakeholder groups, were involved in all the stages of the analysis. These collective processes can enhance validity by promoting reflexivity67 and ensuring that the perspectives of all stakeholder groups are considered in the analysis.68 There was a high level of agreement between stakeholder groups, with 11 features of CRT work identified as important across the groups. These were organised into three broad domains: (1) organisation of care, (2) the content of CRT work and (3) the role of CRTs within the acute and continuing care systems. For each feature, similarities and variations in stakeholders’ views on successful and unsuccessful aspects of current CRT practice and implementation were considered. Findings are summarised below.
To our knowledge, this was the first study to provide an in-depth, multiperspective qualitative exploration of the successes and failures of CRTs across a wide range of contexts, covering trusts in urban, suburban and rural areas of England. The perspectives and experiences of all relevant stakeholder groups were explored and compared. The large number of qualitative data and the diverse demographic, clinical, service use and professional characteristics of the study’s sample help ensure that the common themes identified generally reflect the views of these stakeholder groups. Given that service users were recruited through CRT clinicians, it is likely that the study’s sample under-represents service users who were not very engaged with services or who had generally negative experiences of CRTs. Because of the challenges of reporting concisely on such a large data corpus, the study focused on common themes and prevailing views within each stakeholder group. Thus, some less common concerns or views have not been thoroughly explored in depth, and the approach in this initial analysis of a very large number of interviews is necessarily broad rather than in-depth. Many of the findings from this study had direct implications for the development of a more highly specified CRT model, especially in relation to improving service user and carer experiences. Priority areas for service improvement and for further research in optimising team function and outcomes include continuity of care, provision of emotional and practical support, carer involvement, and the quantity and quality of contact with CRT staff. Findings also highlight challenges to CRT implementation in contexts of stringent resource limitations and complex service configurations. Most previous UK CRT research has been conducted in urban areas; some stakeholders in our interviews raised challenges to implementing the CRT model fully in more rural areas (e.g. in achieving a rapid response service and providing frequent home visits). Further research is needed to understand and assess barriers of, and facilitators to, successful implementation of good CRT care in different contexts. In relation to initiatives for CRT improvement, qualitative research including multiple stakeholder perspectives is a rich source for understanding the impact of initiatives and barriers of, and facilitators to, achieving intended improvements. Use of implementation science methodologies in understanding how to overcome barriers and achieve change is a priority for future research. See Appendix 4 for the published report of this work.69 The report is accessible at the following URL: https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-016-1139-4 (accessed 28 March 2019). Fidelity scales are measures of the extent to which services adhere to a model of high-quality care, based on evidence and on stakeholder consensus. They are designed as tools to support quality improvement.56 There is evidence for some fidelity scales of an association between higher fidelity and superior service outcomes.70 Fidelity scales have been developed for use in some mental health service models,71 especially in the USA, but none exist for CRTs. The aim of this study was to develop a highly defined and validated fidelity scale for CRTs. The study aimed to (1) systemically develop a CRT fidelity scale, (2) test its feasibility and utility in practice and (3) conduct an initial investigation of its psychometric properties. Concept mapping72 was used to construct the CRT fidelity scale. This is a method used in the development of measures to identify and group into relevant domains a set of priority items based on stakeholder consensus. Potential service model characteristics were identified from module 1 (the literature review),58 the national survey of CRT managers63 and a qualitative investigation of CRT stakeholder perspectives. All yielded potential items for inclusion in a longlist of characteristics of high-quality CRT delivery. Next, this longlist of characteristics, identified as associated with good practice in CRTs, was condensed and combined into a list of fidelity scale statements by a group of academic, service user and clinician stakeholders by, for example, removing duplicates and combining closely related items. Following this, a series of concept mapping meetings involving stakeholders with different perspectives was held. Participants were asked to prioritise statements in terms of their importance for CRT best practice and to sort statements into groups of conceptually related statements. Specialist Ariadne concept mapping software (version 1.0; Talcott, Amsterdam, the Netherlands) was then used to analyse participants’ grouping of statements, using a form of cluster analysis, to generate a series of ‘cluster solutions’, which distinguished domains of CRT fidelity to reflect how participants grouped statements. The software also generated a mean importance rating for each statement from participants’ prioritisation of statements, which allowed the fidelity statements considered most important by stakeholders within each conceptual domain to be identified. Cluster solutions generated by the software were then reviewed by a small group of stakeholders, who selected the cluster solution that produced the most conceptually coherent set of domains of CRT fidelity, and named each domain, reaching consensus through discussion. The chosen cluster solution, that is, the final concept map, was then used to develop the CRT fidelity scale. Statements representing each fidelity domain were selected for inclusion in the fidelity scale. Statements of higher importance within each domain were selected for inclusion. Operational definitions were developed for each statement and anchor points specified for scoring, with reference to the development work carried out in module 1. Adherence to fidelity items is measured on a five-point scale. In common with other fidelity scales,70 a score of 5 on each item was taken to represent excellent model fidelity, that is, a service was adherent to best practice standards for this item. A score of 3 was taken to represent moderate fidelity, that is, a service was adhering to some essential requirements for best practice in this area, but not achieving all desirable elements of best practice for this item. The measure was thus able to distinguish services with ‘good’ overall model fidelity (i.e. a mean score of ≥ 4 per item) and services with ‘fair’ model fidelity (i.e. a mean score of 3–4 per item). At each stage of development, further stakeholder consultation was sought. A fidelity review process was developed and piloted for collection of fidelity scale data. Reviews took place over 1 full day and were carried out by three reviewers: one clinician, one service user/carer and one researcher. CRT managers were contacted before the review to identify required documents and to establish the schedule. The review day involved interviews with managers, staff, liaising service staff, service users and carers; reviewing 10 anonymised case records; and reviewing service policies and records. Reviewers collected data using checklists and interview schedules, and scored fidelity items together at the end of the review. A draft fidelity review report was sent to team managers for clarifications before a final report and fidelity score were provided. Pilot reviews were initially conducted in four CRTs, followed by a 75-team review with CRTs from different parts of England, Scotland and Wales (see Crisis resolution team fidelity survey). Following reviews, researchers sought feedback from managers regarding the acceptability of the fidelity review and the validity and clarity of the fidelity scale. Three psychometric properties of the fidelity scale were investigated:
To construct the scale, an initial list of 232 identified service model characteristics was condensed to 72 fidelity scale statements by examining them for duplication and overlap. Sixty-eight participants were then involved in concept mapping, including representatives from all stakeholder groups (including service users, carers, CRT clinicians and managers, and CRT researchers). A four-cluster solution was chosen as the most coherent concept map. The clusters were named as (1) referrals and access, (2) content and delivery of care, (3) staffing and team procedures and (4) location and timing of care. Thirty-nine statements receiving a high level of endorsement were included in the final fidelity scale, representing each of the four clusters. The three broad stakeholder groups in the process were service users and carers, CRT clinicians and others (including CRT researchers). All statements that were rated as very important (i.e. a mean score of > 4) by any of the three groups or as moderately important (i.e. a mean score of > 2.5) by more than one stakeholder group were included in the scale, either as a distinct item or among item-scoring criteria. Table 4 summarises the four clusters of items (subscales) defined through the process and the mean importance score for the items in each. The first version of the fidelity scale was piloted in 75 teams in a range of urban and rural locations and catchment sizes. Feedback indicated that fidelity reviews and reports received afterwards were perceived as acceptable and helpful, although reviews were time-consuming. The first version of the fidelity scale was reviewed after 50 reviews. All 39 items were retained, with minor changes made to 19 items in creating version 2 of the fidelity scale. Version 2 was piloted alongside the original version in nine CRTs. Seventeen reviewers participated in the inter-rater reliability exercise, using version 2 of the fidelity scale. An estimated kappa correlation of 0.65 [95% confidence interval (CI) 0.54 to 0.76] between ratings was identified, indicating moderately high similarity between ratings within an item. The estimated intraclass correlation (ICC) averaged over the 16 raters was very high, at a value of 0.97 (95% CI 0.95 to 0.98). Consistency of agreement ICCs produced nearly identical values (i.e. an ICC of 0.97), equivalent to Cronbach’s alpha. Results indicate that the fidelity scale distinguished between higher- and lower-fidelity CRTs, with a wide range of scores obtained on each item. Piloting version 2 of the scale generated modest changes to fidelity scores compared with version 1 piloting, but increased scale clarity. Using concept mapping, the study systematically developed the first CRT fidelity scale, based on best available evidence and stakeholder input. The so-far inconsistent implementation of CRTs has limited the potential benefits of providing this service model.24 The CORE fidelity scale has potential to support CRT model implementation and aid quality improvement work by providing a clear specification of this service model. This addresses the study’s overarching aim of improving the care provided to service users experiencing an acute mental health crisis. There were two limitations of the study’s CRT fidelity scale development: the scope of stakeholder consultation and the degree of testing of psychometric properties conducted. It was aimed to include stakeholder views at all stages of development. However, during concept mapping, clinicians outnumbered service users and carers in terms of input, and some stakeholder groups (e.g. primary care referrers to CRTs and emergency services) were not represented. Stakeholder input was largely from the UK, so it is not known how far the fidelity scale represents views of international stakeholders and how appropriate it is for use outside the UK. The content and criterion validity of the study’s fidelity scale require investigation to demonstrate the legitimacy of the scale as a measure of service quality, and further testing of the scale’s inter-rater and test–retest reliability is also a priority. A clinical need to develop and evaluate a resource manual to help CRTs achieve service standards and aid quality improvement38 has been addressed in the (module 3) CORE SIP trial.73 There is also a need to explore the feasibility and utility of the fidelity scale in international contexts where the model is used, such as in Norway and the Netherlands. The development of the CRT fidelity scale has demonstrated concept mapping to be a feasible method in developing and defining the CRT service model. The development and piloting of the study’s CRT fidelity scale provides a model of PPI in research and audit.74 As well as serving to pilot the CORE CRT fidelity scale, the 75-team survey provided useful information about implementation of the CRT model nationally in England. See Appendix 4 for a reference to the published results of this work.69 Forty-seven NHS trusts were approached, with 149 CRTs within their catchment areas. Of these, 75 CRTs from 27 trusts took part in the survey. The study sample of 75 teams comprised 70 teams in England, one team in Scotland and four teams in Wales. In the 75 teams surveyed, the total scores ranged from 73 to 151 (possible range 39–195). The median score was 122 and the interquartile range (IQR) was 111–132 (these two findings were not reported in the published paper69 on the development of the scale; see Appendix 4). Higher scores indicate higher model fidelity. The median scores per item across all 75 teams for each subscale within the CRT fidelity scale are provided in Table 5. The low score for the ‘location and timing of help’ subscale reflects the fact that many CRT teams had no access to residential crisis houses or acute day units, and typically provide a less intensive service (i.e. less frequent visits to service users) than recommended in the CORE scale and in the original CRT guidance. The fidelity items for which teams typically showed poor fidelity to the CORE CRT model (median score of 1 or 2) and the items for which teams typically showed good model fidelity (median score of 4 or 5) are presented in Table 6. There are three main findings from this survey regarding implementation of the CRT model in practice in England. First, CRTs typically achieve only moderate fidelity in adhering to an optimal CRT model: teams’ scores ranged from low to moderately high fidelity; no team achieved an average item score of > 4, which would indicate very good model fidelity. Second, the range of scores achieved for each item suggests that the CRT fidelity scale is not unrealistically rigorous: all items were being delivered with high fidelity in some CRTs and, therefore, are achievable in routine care. Third, although areas of strength and limitations regarding CRT model adherence varied from team to team, the study’s survey shows that CRT teams typically struggled to provide core elements of the CRT model specified in government guidance,3 such as rapid response, a range of care from a multidisciplinary team and help with planning recovery and future crisis response. This study thus corroborates the findings of previous surveys that relied on CRT managers’ reports of their teams’ organisation and service delivery,24,63 and suggests that the CRT model has been inconsistently and only partially implemented in England. It suggests that a national mandate and policy guidance may in themselves be insufficient to ensure implementation of a complex service model as planned. It reinforces the need to develop effective resources to support implementation and service improvement in CRTs. A CRT SIP was developed to support teams in improving adherence to the CRT model, as defined in the CORE CRT fidelity scale. See Appendix 5 for the published report of this work.73 The report is also accessible at the following URL: https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-016-1283-7 (accessed 28 March 2019). The CORE CRT SIP was designed to support CRT teams in identifying target areas for service improvement and in developing and implementing plans to improve current practice. It was modelled on the US EBP programme’s implementation resource kits,57 with content informed by module 1 and by the content of the fidelity scale. Development of the SIP was led by a co-applicant (Steve Onyett), who brought expertise in promoting leadership and change management in health settings, and the study principal research associate (Elaine Johnston), who brought experience of working as a clinical psychologist in CRT teams. The SIP’s content was developed and refined iteratively, with input from the study team and consultants, and advice from senior clinicians with relevant CRT experience from local NHS trusts. Structures and resources included in the SIP are available in the online CRT resource pack and are accessible at the following URL: www.ucl.ac.uk/core-resource-pack (accessed 7 March 2019). The structures and resources include:
The CORE CRT SIP, with implementation support from facilitators during the year of the intervention, was thus designed to provide ongoing help to teams during the course of the project. The nature of this support could be individualised to be appropriate to the team context and to address specific local barriers to implementation of the CRT model, as specified in the CORE fidelity scale. This could include support with securing leadership support, structuring team workflows to support implementation or providing expert technical assistance with implementing the CRT model when required, in line with EBP implementation theory.77 During the development and piloting of the CRT fidelity scale (modules 2.1 and 2.2), plans for the CORE CRT SIP trial (module 3) were refined. Consultation on the trial design, outcome measures and intervention content were achieved by the following:
Twenty-five CRTs from eight NHS trusts were recruited to participate in the trial. These were selected to include inner- and outer-London sites, other large urban areas in southern England and more mixed suburban and rural areas. A full nationwide spread of sites was not feasible within the study’s resources. The feasibility and acceptability of proposed outcome measures for the trial were confirmed. Measures of service user experience and assessing satisfaction with, and perceived continuity of, care had been used in the pilot trial of the CORE peer-supported self-management programme (module 5), so were known to be suitable for use with a CRT population. Measures of staff well-being had been previously used successfully in a multisite study of staff morale in acute services, including CRTs, which was also led by the CORE study chief investigator.78 The CRT fidelity scale was thoroughly piloted in the 75-team survey. The research and development and information technology (IT) departments of participating NHS trusts were contacted to check the availability of anonymised acute service use data for cohorts of service users for the trial. Similar data had also been collected for participants in the module 5 pilot trial for the peer-supported self-management programme. Epidemiologists at UCL were consulted about how best to collect data about the catchment area populations of CRTs participating in the trial, from which to assess teams’ admission rates. Population data for 18- to 64-year-olds were available from the Office for National Statistics via ‘nomisweb’ at a broad (borough) or detailed (lower super output area) level as required to allow accurate estimation of the population served by each CRT team (www.nomisweb.co.uk/; accessed 7 March 2019). Meanwhile, the content and format of the online CRT resource pack (designed to support service improvement in the CRT trial) was iteratively improved. Senior CRT clinicians from teams not participating in the trial gave feedback on early versions of the online resource. The staggered start date for CRT teams participating in the trial allowed further early feedback and refinement of the resource in response to teams’ early experience. Developments made through this process included (1) the addition of a ‘news’ section and access to the UCL crisis care Twitter feed (www.twitter.com; Twitter, Inc., San Francisco, CA, USA), to increase the feel of the resource pack as a living, evolving aid for CRTs, (2) standardisation of the way guidance for each fidelity item was presented to aid clarity (the subheadings were target, why this is important, ways of doing this well and examples of good practice), and (3) addition of more case study examples and video testimonies to reinforce the importance of each fidelity item to service users’ experience and to show that they currently are being achieved in UK teams. The full protocol for this trial has been published73 (see Appendix 5). The report is also accessible at the following URL: https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-016-1283-7 (accessed 28 March 2019). A cluster randomised trial was conducted in order to evaluate the impact of the CORE CRT SIP. CRTs allocated to the intervention arm received the SIP over a 1-year period; control CRTs did not receive the intervention. The trial tested the hypotheses that a CRT SIP can improve service users’ experiences of CRT care (primary hypothesis), reduce acute service use and improve CRT staff well-being. The trial also aimed to investigate whether or not fidelity scores increase in CRTs receiving the intervention, and to explore the associations between CRT fidelity scores and service outcomes. Through a qualitative evaluation, the trial sought to understand stakeholders’ experience of the SIP and barriers to, and facilitators of, its implementation. Twenty-five CRTs were recruited from eight NHS trusts across southern England and the Midlands, covering urban, suburban and rural areas. CRTs identified by the earlier CORE fidelity survey as already achieving good model fidelity were excluded, as were CRTs anticipating any major service reorganisation during the trial period. At least two participating CRTs from each NHS trust were required, to allow randomisation of CRTs within trusts. Fifteen teams were randomised to receive the SIP, which involved support from a CRT facilitator, access to online implementation resources and feedback from regular service audits (for further information regarding the intervention, see Module 2.2). Ten teams were allocated to the control group. The primary outcome was service users’ satisfaction with CRT service, as measured by the Client Satisfaction Questionnaire (CSQ).79 Service user satisfaction was chosen as the primary outcome because of the evidence from previous qualitative studies and surveys that the implementation of CRTs has not clearly improved service users’ experiences of mental health crisis care to the extent hoped for, and which trials indicate is possible.58 The specific choice of the CSQ as the primary outcome measure was because it was brief and widely used, and has proved sensitive to differences between service models in other studies in acute care settings.79 The calculated sample size for the trial provided 95% power to detect half of 1 standard deviation (SD) difference between groups in mean satisfaction measured by the CSQ (3.5 points assuming a typical SD of 7.0 points) using a two-sided test, allowing for within-team clustering (ICC 0.05 for clusters of size 15) and unequal 3 : 2 (intervention-to-control) randomisation. Randomisation was stratified by NHS trust to ensure that within each trust some CRTs received the intervention, and some acted as control teams. A statistician, independent of the study, generated allocation sequence lists and conducted randomisations for all participating teams within each NHS trust at the same time, once baseline fidelity reviews had been completed at all sites within the trust. There were no blinding arrangements, as study researchers and CRT staff were aware of teams’ allocation status. Service user participants providing data for the trial’s primary outcome and trusts’ informatics teams providing service use data are unlikely to have known whether or not teams were receiving the SIP. The trial comprised the following components:
Teams were recruited to the study between September and December 2014, with a 1-year trial intervention period. All baseline fidelity reviews were conducted before teams were randomised. Baseline data from service users and staff were collected following each team’s fidelity review; for some teams, baseline data collection was completed after randomisation. Statistical analyses were conducted using Stata® (StataCorp LP, College Station, TX, USA) and qualitative data were analysed using NVivo 9 software. For each of the trial components, the analysis was as follows:
All 25 CRTs were successfully recruited and retained in the trial. Sixty-two per cent of the eligible service users, who expressed willingness to CRT staff to participate in the study, gave informed consent to researchers to complete measures of satisfaction and continuity (353/567 at baseline and 371/594 at follow-up). The mean age of service user participants was 43 years (SD 15 years) at baseline and 42 years (SD 14 years) at follow-up; the majority were female (59% at baseline and 61% at follow-up) and most identified as white (86% at baseline and 81% at follow-up). At each time point, 79% of clinical staff then working in the participating CRTs completed the survey (441/560 at baseline and 432/544 at follow-up). The mean age of staff participants was 43 years (SD 10 years) at baseline and follow-up; the majority identified as female (66% at baseline and 63% at follow-up) and white (68% at baseline and 65% at follow-up). (For further information about recruitment rates and service user and staff participants’ characteristics, see the paper84 referred to in Appendix 6.) No serious adverse events (SAEs) were identified during participant recruitment and data collection, and no study-related harms were reported by participating CRTs during the trial intervention. The key findings are summarised in Table 7. There was no statistically significant difference in service user satisfaction (primary outcome) or service user-rated continuity of care between the intervention and control group teams. There were no significant differences between the two groups in three of the four staff self-rated well-being measures; better general psychological health outcomes for intervention group staff just achieved significance. There were significantly fewer inpatient admissions and inpatient bed-days (including both voluntary and compulsory admissions) at follow-up in the intervention group than in the control group, with adjustment for baseline rates. There were no significant differences between the two groups in the rates of compulsory admissions or rates of re-admission to acute care for people who had already experienced an episode of CRT care. There was a weak correlation between an increase in team fidelity scores over the intervention period and improved patient satisfaction, with a correlation coefficient of 0.34 (p = 0.10). This can be interpreted so that a change of 10 fidelity points corresponds to a change of 0.65 CSQ points. There was a weak correlation between an increase in fidelity scores and a reduction in re-admissions following CRT care (correlation coefficient 0.38; p = 0.06). No significant association was found between the change in fidelity scores and changes in hospital admission rates, inpatient bed-days or staff well-being. For a breakdown of CRTs’ fidelity scores and relationships with outcomes, see the paper84 referred to in Appendix 6. The qualitative evaluation established three broad domains capturing stakeholder views and experiences of the SIP (for a more detailed description, see Appendix 7):
Fairly good response rates of > 60% of service users referred by CRTs and nearly 80% of CRT staff were achieved. The primary outcome was negative, but our results did suggest that the implementation of the CORE SIP may have led to a reduction in hospital admission rates and inpatient bed use, when adjusting for baseline scores. Given the importance of reducing hospital admissions as an aim of CRTs, and to patient experience, it was believed that this positive finding provides an encouraging indication that the CORE CRT SIP is a worthwhile intervention. Although no significant differences were found in other outcomes, there were non-significant trends in favour of the intervention group teams regarding patient satisfaction, re-admission to acute services following CRT care and all staff well-being outcomes. The majority of intervention teams improved model fidelity over the study period, whereas none of the control teams did, which suggests that our programme was successful to some extent. There was also some evidence across all teams of a small positive association between increase in fidelity and improvement in outcomes for patient satisfaction and re-admission following CRT care. The inclusion of 25 CRTs in varied geographical and sociodemographic contexts suggests that the trial findings may have general applicability to CRTs in England; however, generalisability to other national contexts is uncertain. The qualitative interviews show that, overall, stakeholders had positive experiences of the SIP. The majority of stakeholders were able to describe ways in which the programme had a positive impact in their team, resulting in service innovations and improved staff motivation and service user experience. Several limitations arose from the study design. CRT service delivery and outcomes may have been affected by local policy directives and service initiatives that were independent of the trial intervention; the extent of these, and whether or not some teams were affected more than others, is unclear. As stakeholders pointed out, the 1-year follow-up period may have been too short for teams to implement their service improvement plans and to capture all the effects of the intervention. Two limitations regarding the data collected for the primary outcome of service user satisfaction create uncertainty about whether the lack of a positive result on the primary outcome reflects a real lack of impact from the SIP on patient experience or methodological limitations of the study. First, the ICC for the primary analysis (ICC 0.12) was somewhat larger than anticipated (i.e. ICC 0.05), suggesting that the power of the main study was lower than planned (83%). However, the observed reduction in CSQ of 0.97 units was also far lower than the anticipated reduction of 3.5 units, and too small to be very meaningful in terms of patient experience, so this reduced power is unlikely to have substantially influenced the study findings. Second, and more importantly, the study participants may have been unrepresentative. It was not possible to establish from CRT teams how many service users were eligible and how many were approached by staff about taking part in the survey of satisfaction to measure the primary outcome for the study. Therefore, it cannot be determined whether recruited participants were representative of the CRTs’ service users in general or whether they over-represent those who are best engaged with CRT staff. This might also relate to the ceiling effect that was observed with the trial primary outcome measure (i.e. the CSQ). Service user participants were highly satisfied with care compared with previous studies.15,18 In addition, the brief study measure might have not been sensitive enough to capture differences in service user satisfaction with CRTs. Neither CRT staff nor fidelity reviewers were blind to teams’ allocation status during follow-up CRT fidelity reviews. Teams in the intervention group might have been more motivated to demonstrate fidelity gains than those in the control group. Reviewers may have also been subconsciously biased in favour of the intervention group. These phenomena may have served to exaggerate the difference found between intervention and control teams’ fidelity scores. Study researchers were unable to independently verify that all relevant records were provided from NHS patient records, as they were anonymised. One trust was unable to provide information about whether inpatient admissions were compulsory or voluntary at follow-up. Researchers could not determine from the information provided whether service users were physically present on the ward during the whole of their inpatient admissions or whether they were on leave. Outcomes from the SIP that were trialled showed some promise in relation to effects on inpatient admission, but the primary study hypothesis was not confirmed. This suggests that the SIP and the fidelity scale are a potential basis for quality improvement in CRTs, but that further refinement is still needed. Only a weak relationship was found between changes in overall fidelity scores and changes in outcome, and not for all key CRT outcomes. Therefore, the criterion validity of the fidelity scale and its utility as a marker of CRT service quality should be further explored. More robust testing of inter-rater reliability of the scale is needed in vivo. It is possible that the critical components of CRT services lie within the CORE fidelity scale, but are not sufficiently weighted for teams’ overall fidelity scores to relate closely to service outcomes. Future research could further investigate the relationship between team outcomes and specific fidelity scale item or subscale scores. A future economic evaluation could explore the cost-effectiveness of the intervention as delivered in this trial. A process evaluation could helpfully examine which aspects of service provision were addressed in service improvement plans and where additional support may have been needed to enhance the impact of the programme. The types of activity undertaken by CRT facilitators to support teams’ service improvement could be explored from facilitators’ monthly reports to help identify any gaps in support. Workstream 1 of the CORE programme has produced several outputs of practical usefulness to clinical services and service planners in supporting service improvement in CRTs. These include:
The value of these resources to service planners and policy-makers is demonstrated by their use by expert bodies and policy-making organisations nationally in the UK. Data from the module 1 national CRT survey were shared pre publication with Mind, NHS England, Health Education England and the Care Quality Commission (the body responsible for quality inspections of health and social care services in England), and informed publications including Mind’s acute care campaign report ‘Listening to Experience’85 and NHS England’s London Mental Health Crisis Commissioning Guide.86 The CRT fidelity scale is advertised as a ‘national inspiration’ in the Crisis Care Concordat campaign led by NHS England.87 Benchmarking data from our 75-team fidelity survey have been used by the Care Quality Commission88 and the Royal College of Psychiatrists89 in recent reports presenting recommendations for CRT care. Resources from the online CRT resource pack have been shared with the Royal College of Psychiatrists’ Home Treatment Accreditation Scheme to support service improvement among CRTs in the Home Treatment Accreditation Scheme network. The CORE SIP trial (module 3) did not achieve all its aims, but showed promise in helping to reduce inpatient service use. Two major challenges for future research remain. First, there were limited relationships found between CRT teams’ fidelity gains and improvement in outcomes in the trial: there were weak to moderate associations between fidelity gains and improved patient experiences and reduced re-admissions following CRT care, but not with inpatient admission rates or bed use. Perhaps the lack of clearer associations is understandable; teams’ fidelity changes were typically modest in the trial, and most teams were of moderate fidelity at both trial time points. Experience from other fidelity scales suggests that even when a well-established fidelity measure is paired with a robust, relevant outcome measure, the correlations are in the moderate range at best.70,90 However, for CRTs, further research is needed to establish which aspects of CRT model fidelity might relate most closely to specific outcomes. The study did not find empirical confirmation that the total score on the CRT fidelity scale is a wholly valid indicator of service quality. Second, further research is required about how best to support service improvement in CRTs. The results from the CORE trial offer mixed messages about the impact of the CORE SIP. The trial’s primary outcome was not significant, suggesting that the programme did not improve patient experience in CRTs. The proximal goal of fidelity gain was achieved for most teams (and in none of the control services); some important secondary outcomes (inpatient admission rates and bed use) did yield significant, positive results. Further evaluation of a set of service improvement resources is desirable, which address the implementation challenges identified in the qualitative evaluation of our trial (e.g. leadership support, underuse of the online support, ensuring available time for the CRT facilitator). Evaluation of the impact of such a set of resources on patient experience in a patient population that is more clearly representative (and probably less satisfied) than the service user participants recruited to our trial is also desirable. The CORE CRT SIP does provide useful resources for future studies to build on, and which are of immediate practical use to support CRTs in developing their own quality improvement plans. It was the aim within workstream 2 to develop and evaluate a peer-supported self-management programme to address the gap between CRT support and continuing care, promote recovery and reduce relapse. An audit provided to the research team by one CRT participating in workstream 1 (see Module 2) found a 50% rate of re-admission to acute care following a period of CRT support. This high rate of relapse following CRT care has since been confirmed by a multisite study.91 In addition, it was found in workstream 1 that interventions to prevent relapse and re-admission to acute care are generally not part of CRT service delivery. Mental health self-management programmes have the potential to address these limitations by supporting service users to learn skills to manage their psychological well-being and avoid future crises.92 Self-management programmes can be supported by health professionals, health para-professionals and peer-support workers (PSWs) who have themselves experienced mental ill health, or a combination of these groups. Government health policy and guidelines93–95 promote the use of PSWs to provide support and some specific interventions to mental health service users. Support provided by PSWs has been advocated as a way to improve outcomes for people following a mental health crisis.96 Guidelines and consultation with advocacy groups suggest that peers may be appropriate supporters for the delivery of self-management programmes, whereas the employment of PSWs to deliver self-management support is becoming increasing common within the NHS.97,98 However, existing evidence for the clinical effectiveness and cost-effectiveness of peer-support programmes is still inconclusive, and further trials of clearly defined interventions are needed.99 To our knowledge, there have been no evaluations of peer-facilitated, self-management interventions for people leaving CRT care. In the study’s initial consultations with service users and carers, clinicians and trial methodologists in preparation of this proposal, PSWs emerged as the preferred choice for delivery of the self-management intervention. Thus, the CORE research group set out to develop and evaluate a peer-supported, self-management intervention. The main goals of the intervention were to (1) reduce relapse, re-admission and bed use rates by enhancing service users’ and their networks’ abilities to respond to early signs of crises; (2) enhance continuity by bridging the perceived gap between crises and continuing care; and (3) enhance service users’ abilities to manage their illnesses and progress towards recovery goals. Workstream 2 reports the iterative development of a peer-supported, self-management intervention for people leaving CRT care, following Medical Research Council (MRC)’s guidelines100 for developing and evaluating complex interventions. This comprises:
A report based on this review was published by the National Institute for Health and Care Excellence (NICE) as guidelines,95 which were subsequently updated. See Appendix 8 for a link to the published paper.101 The report is also accessible at the following URL: www.nice.org.uk/guidance/cg178 (accessed 7 March 2019). The review is registered with PROSPERO as CRD42017043048. The study aimed to review the evidence of short-, medium- and long-term clinical effectiveness of self-management programmes for SMIs through a systematic meta-analytic review compared with other comparable therapies or standard care. Quantitative synthesis of results from different studies was also conducted. Studies were included if participants were adults, aged ≥ 18 years and diagnosed with a SMI, that is, with a clinical diagnosis of schizophrenia spectrum disorders (schizoaffective disorder, delusional disorder and psychosis), bipolar disorder or major depression. Studies with mixed populations (including personality disorder) using secondary care mental health services were also included. Participants at any stage of their illness, that is, as part of early intervention or more chronic/later stages, were included in this review. Further details and full search terms are provided in the paper101 referred to in Appendix 8; searches were conducted without time limit or geographical restrictions. The types of self-management programmes included were those that aimed to help service users develop their own mental health self-management strategies and coping skills in order to achieve better clinical and/or social outcomes. The outcomes of the included studies were grouped as symptom-focused outcomes, service use outcomes, functioning outcomes and recovery-related outcomes. Characteristics of the study design, the intervention, participants and outcomes for all available data at all provided time points were extracted. Authors were contacted and asked to provide any missing data. Studies were reviewed for bias using the Cochrane risk-of-bias tool,102 which was used to assess the quality of evidence by evaluating the risk of bias, inconsistency, indirectness, imprecision and publication bias. Extracted quantitative data were meta-analysed using appropriate random-effects models for dichotomous data (relative risk and 95% CIs) and continuous data [standardised mean differences (SMD)]. Outcomes were analysed at the end of treatment, post intervention (short term), up to 1 year post intervention (medium term) and over and including 1 year post intervention (long term). When more than one self-management intervention was evaluated in a study, these arms were combined for comparison with the control group. Review Manager software (RevMan 5.2; The Cochrane Collaboration, The Nordic Cochrane Centre, Copenhagen, Denmark) was used to conduct the meta-analyses. When outcome data were reported for more than one follow-up point, the time point closest to 1 year post intervention was used. Thirty-two RCTs with 4961 randomised individuals met the eligibility criteria for this review. The types of interventions included typically offered guided self-management, supported by a clinician or peer worker. There was wide variation in the duration of programmes, and in the number, duration and frequency of sessions within programmes. The majority of studies were conducted in Western countries, with a smaller but significant proportion in Asian and developing countries (κ = 7). The majority of studies included participants who were currently living in the community, with eight studies recruited from inpatient settings. In relation to clinical diagnosis, 16 studies included participants with only schizophrenia spectrum disorder and seven included only those with a diagnosis of bipolar disorder. The remaining nine included mixed populations of participants with schizophrenia, psychosis, bipolar, major depressive disorder and personality disorder. The meta-analysis found some evidence of short- and long-term positive impacts, albeit relatively small ones, of self-management programmes on total symptoms (SMD experimental vs. control at follow-up: –0.81, 95% CI –1.18 to –0.44; p < 0.00001), social functioning (SMD –0.55, 95% CI –0.92 to 0.17; p < 0.0000), self-rated recovery (SMD –0.24, 95% CI –0.36 to –0.11; p = 0.0003) and quality of life (SMD –0.26, 95% CI –0.39 to –0.13; p < 0.0001). Initial analyses found no clear effect on number of relapses or length of time in hospital; although heterogeneity was high in these meta-analyses, numbers were small and there was some evidence of an effect on relapse, with one study moved to reduce heterogeneity. Using a broad inclusion criterion for SMI in this review, and including studies of mixed populations of secondary mental health service users, has ensured that the findings of the review are relevant to the CRT context of the workstream 2 CORE study trial. All studies included in this review were RCTs, but there was variation in the reporting of sequence generation and allocation concealment and, as is common in complex interventions, the blinding of participants and personnel was not always consistent. The greatest cause for concern was the selective reporting of outcomes, which was noted or not clearly reported in over half of the studies reviewed. Furthermore, the small number of studies and small number of participants in the studies meant that it was difficult to discern any evident publication bias. Other limitations were high heterogeneity in many of the meta-analyses conducted, and relatively infrequent inclusion of re-admission and other service use outcomes. Comparisons of self-management with treatment as usual and those with active controls were pooled together. This potentially makes interpretation of results more challenging, although, in fact, we found no evident differences in results relating to type of comparator. There were also variations in the timing of the follow-up comparison. The review does not determine the relative clinical effectiveness of different types of self-management programme; further research and an established typology of self-management programmes are needed to achieve this. Importantly, future research needs to be of higher quality. Additional design features should also be considered, for example collecting longer-term follow-up data and applying consistency to the spread of outcome measures, with relapse and service use outcomes consistently measured. Future studies should also examine whether or not any variations in delivery could have an impact on service user outcomes and costs. Establishing whether or not guided self-management programmes (e.g. supported by clinicians or peer workers) are more effective than self-guided programmes is clearly relevant to determining the most cost-effective types of intervention. There is also scope for researching the application of computer-based programmes or applications (‘apps’), as this is a growing area of health research because of the potential for wider reach and reduced costs. See Appendix 9 for the published report of this work.103 The report is also accessible at the following URL: https://bmcpsychiatry.biomedcentral.com/articles/10.1186/1471-244X-14-39 (accessed 28 March 2019). Comprehensive evidence has not previously been available about how effective peer-support programmes are and whether or not they improve outcomes for people with severe mental health problems. Although some systematic reviews of peer support are available, these are limited to subtypes of peer support or for people with depression only. This study aimed to review trials of community-based, peer-provided support for people with SMI. The effects of peer-provided interventions were examined via objective outcomes, such as employment and re-admission to hospital and psychiatric symptoms, as well as self-reported outcomes, such as quality of life, recovery, empowerment, hope and satisfaction with services. Eligibility criteria were established. Published and unpublished RCTs, including cluster RCTs and factorial RCTs, were included. Studies were included if participants were adults with SMI, specifically bipolar or schizophrenia spectrum disorder, or mixed populations of people using secondary mental health services. Studies were excluded if participants were diagnosed with unipolar depression or personality disorder. Peers either were currently using or had previously used secondary mental health services. The review included interventions that involved community-based peer support designed specifically to facilitate recovery from SMI. Residential and inpatient peer-run programmes were excluded, as well as peer-support programmes focusing on outcomes other than mental health recovery. Peer-support programmes were categorised into three types based on previous literature: mutual support groups, peer-support services (additional or separate from standard mental health care) or peer-provided mental health care (see the paper103 referred to in Appendix 9 for further details). The study searched the Cochrane Central Register of Controlled Trials (CENTRAL), CINAHL, EMBASE, MEDLINE, preMEDLINE and PsycINFO from inception to January 2013, with no geographical restrictions, combining synonyms for SMI, peer support and RCT using the AND command. Data were extracted independently by two reviewers. Full search terms are provided in the paper103 referred to in Appendix 9. The SMD and Hedges’ g were calculated for continuous outcomes, and studies were weighted using the inverse of variance. For dichotomous outcomes, risk ratios were calculated and studies were combined using the Mantel–Haenszel method. All outcomes were reported with 95% CIs using random-effects models. Statistical heterogeneity was assessed by inspecting forest plots, conducting the chi-squared test and calculating the I2-statistic. Meta-analysis was conducted using RevMan, and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system was used to rate confidence in the evidence from each analysis of pooled data. Eighteen trials (including 13 from the USA), with a total of 5597 participants, were included. Programme content and participants’ characteristics varied substantially, and outcomes were incompletely reported. Independent reviewers found a high risk of bias. It was only possible to synthesise a small number of studies. The quantitative data synthesis showed some evidence that peer support was associated with positive effects on empowerment from mutual support (SMD –1.44, 95% CI –2.79 to –0.09), and on recovery (SMD –0.24, 95% CI –0.39 to –0.09) and hope (SMD –0.14, 95% CI –0.27 to –0.02) from peer-support interventions post treatment and at follow-up. However, these effects varied across different types of peer support and confidence and this evidence was graded very low. There was very little or no evidence to suggest that peer support was associated with positive effects on satisfaction with services, overall symptoms or hospitalisation. Through using a broad inclusion criterion for SMI in this review, and including studies of mixed populations of secondary mental health service users, it was ensured that the findings of the review are relevant to the CRT context of the workstream 2 CORE study trial. Searches were highly sensitive, and the review provides a comprehensive synthesis of outcomes that were considered important by triallists and service users. The main outcomes were identified a priori, although hope and empowerment were added post hoc. Only randomised trials were included, as they provide the best evidence of efficacy, in line with best practice guidelines for systematic reviews.102 Nevertheless, the conclusions were limited by the small number of eligible studies, which did not allow the conduct of subgroup analyses. Another limitation was the substantial variation in programme content and trial populations, which may have contributed to observed heterogeneity. Confidence in existing evidence was low because of poor study design and incomplete reporting of included studies. Peer support was categorised based on previous typologies, as there were no clearly defined models or fidelity criteria. Overall, there was not enough current evidence to conclude that peer-support interventions are either effective or ineffective for people with severe mental health problems. This review highlighted the need for high-quality research to accurately evaluate the clinical effectiveness and cost-effectiveness of peer-support interventions for people with severe mental health problems. Further research is needed to develop, clearly describe and test theory-based interventions. It was difficult to distinguish different types of peer support using an existing typology as programmes varied in many factors. A distinction between structured self-management-focused programmes and longer-term, less structured ones could be considered in future typologies. Future research should develop and describe clear models of peer support in order to facilitate future trials and help future reviewers synthesise the data more effectively. Trials including a process evaluation could help understand whether or not the peer-support programme is implemented as intended and mechanisms by which any treatment effects are achieved.104 Future research should also examine critical ingredients that might contribute to programme success and appropriate target populations. Most of the studies included in this review were poorly reported and failed to adhere to the CONSORT guidelines for conducting and reporting trials.105 This review further highlights the need to improve standards for reporting trials of social and psychological interventions. Future trials of peer-support programmes must be registered in advance,106 publish their protocols, clearly state all outcomes and time points to be assessed, and describe interventions in full. See Appendix 10 for the published report107 of this work. The report is also accessible at the following URL: https://bmcresnotes.biomedcentral.com/articles/10.1186/s13104-017-2900-6 (accessed 28 March 2019). Although self-management peer-support programmes have shown promising evidence of effectiveness, the empirical evidence for their effectiveness is still limited, as shown by the systematic reviews we conducted (see Module 4.1 and Module 4.2). Following on from these reviews, it was aimed to identify or develop a peer-supported, self-management programme appropriate for people discharged from CRT care. The intervention was selected and developed through consultation with expert reference groups of CRT staff, as well as service users and carers. Multistage qualitative interview and focus group consultations were held to examine programme acceptability and feasibility and determine programme content. At each stage, consultations were conducted by a combination of service users, carers, clinicians and academic researchers. Focus groups and interviews were audio-recorded and then transcribed to be analysed. Initial semistructured interviews explored service users’ views on peer support and priorities for support following CRT care. Semistructured interviews were conducted, usually by service user or carer researchers, with 41 service users recruited from CRTs in 10 NHS trusts in England. A self-management intervention (i.e. the Personal Recovery Plan)108 was then identified by the CORE service user, carer and clinician advisory groups as the best potential fit for the study’s objectives and to address the priorities of interviewed peer supporters. The Personal Recovery Plan108 structure was aligned to the themes reported in our systematic review findings (see Module 4). Focus groups of the key stakeholders were conducted to address the design of the proposed peer-facilitated self-management programme. Further modifications were identified through a feedback session with the newly recruited PSWs, as well as reviews by members of the study team, including CRT senior clinicians. For modelling purposes, a pre-pilot of the intervention was facilitated by four trained PSWs, recruited from within the CORE service user advisory group, with 11 CRT service users post discharge. A psychologist supported weekly group supervision with the PSWs and was available if additional support was needed. Following completion of the one-on-one pre-pilot intervention, qualitative feedback was obtained through a focus group with the PSWs (n = 4) and individual interviews with the service users (n = 9). Both groups provided qualitative feedback on their experiences of the programme and suggestions about how it might be improved. Further qualitative feedback was gathered from participants in the treatment arm of a pilot RCT embedded in the crisis resolution team service (see Module 5). Through these stages of consultations and feedback, a manualised peer-supported intervention for recovery and relapse prevention following a mental health crisis was developed and refined. Qualitative data from interviews were analysed by a team of researchers using thematic analysis. NVivo 9 software was used to code and analyse the data into different themes. The analytic strategy combined inductive and deductive approaches in order to explore initial research questions, as well as themes that characterised participants’ own experiences. A collaborative and iterative approach was adopted: a research team lead by a principal analyst read transcripts, developed a coding frame and regularly discussed emerging themes. This process was repeated at all qualitative phases. In addition, the feedback obtained from the stakeholders and pilot participants was later incorporated into the design of the intervention. Data from initial qualitative interviews and focus groups with CRT staff, service users and carers indicated support for the overall purpose and content of the programme, and for PSWs as providers of the intervention. Service users’ expressed needs were in line with the self-management approach. They endorsed sessions on relapse prevention, crisis planning, signposting and goal-setting as potentially useful. A small number of negative views, mainly concerns raised by carers and clinicians regarding peer support, highlighted the need for adequate PSW training, protocols, supervision and recruitment procedures. Feedback regarding the concerns raised about the programme guided the development of training and procedures for the pre-pilot. Participant feedback after testing in the pre-pilot was largely positive regarding the facilitation of the programme by PSWs. Nearly all participants stated that they benefited from the peer-support experience. Some reported that the relationship itself was more useful than the structured section of self-management workbook, which was underutilised. The workbook was modified based on qualitative feedback from both participants and PSWs in order to improve usability. Minor changes to the language in the workbook were made in discussion with the PSWs and the study service user and carer expert reference groups. By replacing a definition of recovery in the original plan, space was created in the study version for participants to record their personal meaning of recovery, following discussion with their PSW. More blank space was included throughout the plan to allow participants space to include information, potentially using a range of media, relevant to their personal recovery plans. Moreover, additional training for PSWs was provided to facilitate workbook use. In some cases, the peer-support programme could not be adapted to accommodate service users’ views and requests during feasibility testing and piloting. A few participants wanted more than 10 sessions with the PSWs; however, this could not be changed because of funding and requirements for a consistently delivered trial intervention. One service user expressed a wish for a more closely ‘matched’ peer, but it was not feasible to provide this given the small PSW teams at each site. Overall, however, the study has employed a process of systematically and iteratively obtaining stakeholder feedback at the different stages of intervention development. This has helped develop an intervention that matches on the needs and priorities of the target client group. The resulting trial intervention is clearly defined and manualised to enable consistent implementation and replicability. The different stages of consultations were important to strengthen the robustness and usability of our programme. The consultations allowed us to gauge stakeholder acceptability, but they were also useful in identifying gaps and areas of key importance when training PSWs and integrating the programme into a crisis team setting. As Lovell et al.109 pointed out, there are both advantages and disadvantages in using an approach that develops theory and models before conducting an exploratory trial. Developing complex interventions is a time-consuming and resource-heavy process, and incorporating and balancing the viewpoints of different stakeholder groups can be labour intensive. Nevertheless, this process is essential as it systematically incorporates the evidence base with current viewpoints while iteratively testing and refining the intervention design as it develops. The effectiveness of this peer-supported intervention in terms of its impact on service user outcomes and the economic impact remains to be piloted (module 5) and then evaluated in a full-scale definitive RCT (module 6). The pilot trial was used to test out trial procedures regarding recruitment, randomisation, data collection and storage, the study outcome measures and analysis plans. Qualitative feedback from study participants and PSWs was also sought to identify any further changes needed to the self-management workbook or arrangements for training, supervising and supporting the PSWs. This would allow amendments to the protocol to be made, if necessary, before the main trial to optimise the intervention provided to participants and its evaluation. Forty participants were recruited from one CRT in an inner-London NHS trust and randomised to a treatment or control group. Twenty-one participants in the treatment group were offered up to 10 sessions with a PSW who supported them in using a self-management workbook. Nineteen participants in the control group were sent the self-management workbook but no additional guidance or peer support to help them use it. Patient-reported outcome measures were collected at baseline and 4-month follow-up interviews. Data regarding service use, including the primary outcome of re-admission to acute care, were collected from electronic patient records for the year before trial entry and the 1 year of follow-up. Brief qualitative interviews were conducted with participants in the treatment group at the end of the trial intervention to explore their experience of the programme of support. A focus group was conducted with the PSW providers of the intervention about their views on content and implementation of the programme. The few changes made to the trial protocol between the module 5 pilot trial and the module 6 main trial are reported in Module 6, Methods. Otherwise, the trial methods were retained as described in the section below on the main trial (see Module 6) and in the paper110 referred to in Appendix 11. Of the 234 patients screened, 151 met the trial eligibility criteria. Of these 151, 40 (26%) were recruited to the trial. Fifteen could not be contacted, 42 declined and no decision was elicited from 54 participants before 1 month following CRT discharge had elapsed and they became ineligible. Of the 21 participants randomised to the treatment group, 17 received the trial intervention as per protocol (minimum of three sessions), one declined to meet their PSW and three were allocated to a PSW late following administrative oversights in the study or clinical teams. The 4-month follow-up interviews were completed with 34 out of 40 participants (85%). Primary outcome data regarding re-admission to acute care were obtained for all participants. Follow-up interviews were completed with 18 out of the 21 participants (86%) who were randomised to receive PSW support. The other three participants declined a follow-up interview. A focus group was conducted with PSWs (n = 4). Most feedback about the programme was positive. Participants commented on the usefulness of PSWs providing information about mental health problems and treatment, and of having additional support. Eight participants commented on the helpfulness of meeting someone with similar symptoms or experiences, and 11 out of 16 participants (69%) who met with a PSW described the relationship as good:
Help with practical activities was also valued, such as looking for local services, coping strategies for tinnitus, financial help and help with housing. Positive comments about the recovery plan were that it is useful to have goals and that it was useful for recovery and thinking about triggers or keeping well:
One participant liked the blank pages for ‘paintings’, but another did not like having blank pages. Four participants did not find all the sections ‘relevant,’ did not find it useful or were not ready to think about recovery. The PSWs (n = 4) suggested no further changes to the recovery plan in their focus group. Participants reported that sessions typically lasted for 1 hour and that this seemed a good length. The meetings were in various locations including participants’ homes, community settings and a crisis house. Four participants found the sessions unhelpful: one did not like weekly meetings, one did not get on with her PSW or like working to a timetable, one was made homeless and was disappointed that his PSW could not help with this, and one felt ‘let down’ by her PSW having one session and then becoming unwell. Two other participants found the programme helpful overall, but did not like particular aspects. One participant found recovery-focused work unhelpful when they had not yet met with a psychiatrist. The other participant found PSW advice unhelpful. The most common suggestion for improvements to the programme was to have more sessions (i.e. from four participants). Other suggestions were a participant meet-up group or website, PSWs contacting participants at set times, clarifying whether or not PSWs can meet with participants after the programme, more support with housing problems and recruiting PSWs who are likely to stay well. The pilot trial demonstrated that recruiting and retaining participants in a trial and to the trial intervention were feasible, and confirmed the acceptability of the trial intervention. From the experience of the pilot trial, the following changes were made to the trial procedures and outcome measures for the main trial:
Because primary outcome data were obtained for all participants and no major changes were made to the trial intervention, a decision was taken (with approval from the independent Trial Steering Committee, the Trial Data Monitoring and Ethics Committee and the funders) to incorporate the pilot trial into the main trial. This turned the pilot trial in to an internal pilot: for this reason, outcomes data from the pilot trial are not reported separately, but included in the module 6 main trial results. See Appendix 11 for referral to the protocol of this trial.110 The report is also accessible at the following URL: http://bmjopen.bmj.com/content/7/10/e015665 (accessed 28 March 2019). For a full report of this work, see the paper115 referred to in Appendix 12. The aim of the study was to test the peer-facilitated self-management intervention that was developed (see Module 4.3) to bridge the gap between crisis and continuing care, reducing relapse and promoting recovery. This trial compares the clinical effectiveness and cost-effectiveness of this intervention, delivered by trained PSWs to people just discharged from the CRT, with treatment as usual, supplemented by distribution of the recovery plan booklet by post without any specific support in engaging with it and using it. The primary hypothesis was that service users receiving the experimental intervention would be less likely to relapse within 1 year of discharge from the CRT (indicated by re-admission to any mental health acute care service) than those receiving the control intervention. The secondary hypothesis tested whether or not being in the experimental rather than the control condition is associated with a longer time to first admission to acute care and fewer days in acute care over 1 year. In addition, the study tested the hypotheses that at the 4- and 18-month follow-ups the experimental intervention would result in better perceived recovery, greater self-management skills, greater satisfaction with services, fewer symptoms, less loneliness and better social networks than the control intervention. The study also evaluated the cost-effectiveness of the peer-facilitated self-management intervention compared with the treatment-as-usual intervention at 18 months. This trial was a rater-blind RCT with two parallel arms (with an allocation ratio of 1 : 1) powered on the primary outcome, with adjustment for clustering by PSWs. Participants were identified from the caseload of CRTs in six NHS trusts (four in London, one in South East England and one in the South West of England). Service users were eligible if they were on the caseload of one of the participating CRTs for at least 1 week because of a mental health crisis, and if they had the capacity and willingness to give informed consent to enter the trial and did so within 1 month of CRT discharge. CRT clinicians initially identified potentially eligible participants. The CRT clinicians passed on details to the research team of those who appeared to have capacity to enter the trial and who were willing to be contacted by a researcher. People presenting too high a risk to others to be seen alone even on NHS premises, those discharged to addresses outside the CRT’s catchment area and those who could not understand the intervention when delivered in English were excluded. Any service users who were aged < 18 years were also excluded. The aim was for study conclusions to be generalisable to the whole population of people discharged from CRT and, therefore, restricting sample entry on variables such as age and diagnosis was avoided. At the pilot stage, people with psychosis and bipolar disorder had proved harder to recruit than people with non-psychotic disorders. Therefore, to ensure representativeness, a threshold was set of at least 50% of participants in total to be identified at screening as having schizophrenia, other psychosis or bipolar disorder. A sample size of 440 participants was required to detect a difference in admission rates during the follow-up period of 50% in the control group versus 35% in the intervention group, with 80% power, 5% significance and 1 : 1 allocation. This calculation allowed for clustering by PSW in only the intervention arm, assuming an ICC of 0.03 and a cluster size of 11. The peer-facilitated self-management intervention was delivered in a series of up to 10 sessions with a PSW. The intervention was structured around the completion of a self-management workbook with the following structured components: personal recovery goals, plans to re-establish community functioning and support networks following a crisis, identifying early warning signs and creating a relapse prevention plan, and strategies and coping resources to maintain well-being. In the control condition, participants were sent the self-management workbook to complete by themselves or with others if they wished. The primary outcome was whether or not, in the 1 year of follow-up from study entry, participants were re-admitted to an acute care setting, including acute inpatient wards, CRTs, crisis houses and acute day care. Secondary outcomes included service use measures over a 1-year period (days in acute care and time to first relapse) and outcomes measured at baseline and at the 4- and 18-month follow-ups: self-rated recovery (as measured by the Questionnaire about the Process of Recovery116), self-management skills (as measured by the Illness Management and Recovery Scale92), overall satisfaction with mental health services (as measured by the CSQ79), symptom severity (as measured by the Brief Psychiatric Rating Scale117), loneliness (as measured by the University of California’s, Los Angeles Loneliness Scale111); and social network size (as measured by the Lubben Social Network Scale112). Further demographic and clinical measures were used to characterise the sample and adjust for variables known to be associated with the primary outcome. These include clinical diagnosis as recorded on electronic records, a measure of social circumstances (as measured by the Social Outcomes Index118), a measure of neighbourhood social capital (as measured by the Health and Lifestyles Survey Social Capital Questionnaire113) and self-reported screening measures of alcohol use [as measured by the Alcohol Use Disorders Identification Test – Consumption (AUDIT-C119)] and drug use [as measured by the Drug Abuse Screening Test – 10 items (DAST-10120)]. The EuroQol-5 Dimensions, three-level version (EQ-5D-3L) data were also collected at baseline and at 4 and 18 months to calculate quality-adjusted life-years (QALYs). QALYs were calculated as the area under the curve adjusting for baseline using linear regression. Once written consent was obtained, a study researcher completed baseline measures with all participants as a structured interview. Participants were then block randomised into treatment and control groups, stratified by site. At the 4- and 18-month follow-ups, researchers again sought written informed consent from participants before completing the follow-up research interviews. Researchers completing the assessments were blinded to participants’ allocation status. If for any reason a face-to-face interview was not feasible, a telephone interview was offered, although in that case symptom severity could not be measured as the measure depends on observer ratings. Once all participants from a participating NHS trust were recruited into the study, a study researcher contacted the appropriate administrators or informatics team within the trust to collect data from electronic patient records, using a standardised schedule of the information required for each patient. This procedure was repeated 1 year after recruitment to collect data for primary and secondary outcomes. The NHS IT and administrator staff collecting the data, as well as study researchers cleaning the data and calculating the study outcomes, were blinded to participant group allocation status. All data were entered using a web-based system. Data from electronic patient records were first checked, cleaned and calculated in Microsoft Excel. All analyses were on an intention-to-treat basis and were conducted using Stata. For the primary outcome (re-admission to acute care during the study period), data were analysed using logistic regression with random intercepts, adjusted for clustering by PSW. Diagnosis (psychosis or bipolar disorder vs. no psychosis/bipolar disorder) and centre were entered into the model as fixed effects. For secondary outcomes, linear- or zero-inflated negative binomial regression with random intercepts was utilised when appropriate, with PSW as the random effect, controlling for the baseline value of the outcome, condition and centre. Random-effects Poisson regression was planned to assess the total number of days spent in acute care and Cox regression frailty model for time to first re-admission. However, on seeing the structure of the data, zero-inflated negative binomial with robust standard errors was more appropriate, given that more than half of participants had not spent any days in acute care since baseline. For number of days in acute care, the centre was entered into the model as a fixed effect. Supportive analyses using analogous statistical methods to the primary analyses were conducted on the primary outcome, adjusting for any significant baseline differences between randomised groups and the amount of improvement for both groups between baseline and follow-up. Predictors of missingness were also explored and controlled for in supportive analysis (see Report Supplementary Material 2). The primary cost-effectiveness analysis is a complete-case analysis of the incremental cost per QALY gained of the peer-facilitated self-management intervention compared with treatment as usual. Costs include the cost of the peer-facilitated intervention including cost of training and supervision as well as unit costs applied to mental health service use collected from mental health trust files. Costs are from baseline until 12 months, adjusting for costs 12 months prior to baseline. QALYs between 12 and 18 months were discounted at a rate of 3.5%. All analyses include clustering by PSW. Cost-effectiveness acceptability curves (CEACs) and 95% CIs were calculated using bootstrapping. Further details of the methods and results of the trial economic analysis are provided in Appendix 13. From the 3288 service users screened, 1848 were assessed as eligible and a total of 441 participants were recruited, including participants recruited for the pilot trial. The CONSORT flow diagram (Figure 2) shows participant flow through the study. Forty participants were recruited in the internal pilot trial between 14 May 2013 and 12 November 2013. The remaining 401 participants were recruited between 12 March 2014 and 3 July 2015. Follow-up interviews were arranged as soon as possible after 4 and 18 months from the date of recruitment. The final follow-up interview took place on 23 February 2017. Participants had a mean age of 40 years (SD 13 years), 40% of those who gave information about their gender were male (175/438) and 65% identified as being white (285/438). At the 4-month follow-up, 344 participants completed the interview assessment (completion rate 78%); 255 participants completed the 18-month interview assessment (completion rate 58%). For a breakdown of participant recruitment numbers and further details on participant characteristics (see Report Supplementary Material 1). Findings on the primary outcome favoured the intervention group: 29% of intervention group participants were re-admitted to acute care within 1 year compared with 38% of control group participants [odds ratio (OR) 0.66, 95% CI 0.43 to 0.99; p = 0.0438] (Table 8). This persisted with planned adjustments. Regarding secondary outcomes, time to re-admission was longer in the intervention group than in the control group. However, the numbers of days in acute care were not significantly different. Initial descriptive analyses found an unexpected difference between groups in the number of days between randomisation and discharge from the index acute care admission. In view of this, a post hoc sensitivity analysis was conducted excluding these days and including only days that were part of acute care re-admissions. The intervention group participants had fewer acute care days than control group participants, but not at a statistically significant level [experimental vs. control group: median number of days in acute care 0 vs. 0 (IQR 0–10 vs. 0–22); mean number of days in acute care 13 vs. 19 (SD 31 vs. 40); incidence rate ratio (IRR) 0.90, 95% CI 0.66 to 1.23; p = 0.515]. There was no evidence of a difference between groups in the number of contacts with the community mental health teams, which were the main secondary care providers after CRT discharge. At the 4-month follow-up, overall satisfaction with mental health care received was greater in the intervention group than in the control group. This was because there was a difference in mean satisfaction scores between the randomised groups at 4 months [experimental vs. control: mean 26 (SD 5) vs. mean 24 (SD 6)], which is not shown in Table 8 because of the median (IQR) being more appropriate for skewed data. In addition, there was a significant difference in self-rated recovery favouring the intervention, but this just failed to reach statistical significance in the sensitivity analysis with adjustment for predictors of missingness (IRR 2.18, 95% CI –0.72 to 5.08). There were no other significant differences at 4 months. At 18 months, there was little evidence of any effect, except that the difference in social network size favoured the intervention approached, but did not reach, statistical significance. Similar numbers of participants in each group reported that they had read the workbook at the 4-month interview [i.e. 133 (84%) out of 158 participants in the control group vs. 142 (88%) out of 162 in the intervention group]. However, more participants in the intervention group reported using it to make written plans. Using the workbook to make written plans was reported by between 28% (38/138 participants for the section ‘Managing your ups and downs’) and 44% (61/138 participants for the section ‘Moving on after a crisis’) of participants in the control group, compared with between 58% (83/144 participants for the section ‘Your goals and dreams’) and 64% (92/144 participants for the section ‘Keeping well’) of the intervention group. Serious adverse events were actively monitored for both trial groups until the completion of the 4-month follow-up interview. PSWs and supervisors were asked to report any SAEs directly to the trial manager or data officer and record them on participant contact logs. The trial data officer also contacted the PSWs’ supervisor and CRT manager or principal investigator at each site monthly, who screened for SAEs among participants with reference to patient records. Additional adverse events reported to the study team were also recorded for participants at any stage of participation in the study. Seventy-one SAEs were identified in the pilot and main trial (29 in the treatment group and 42 in the control group):
All SAEs were assessed by involved clinicians, the trial clinical reviewer (the chief investigator) and the independent clinical reviewer (Trial Steering Committee chairperson). Participant deaths and the attempted murder were additionally reviewed by the Data Monitoring and Ethics Committee chairperson. No SAEs were judged to be related to the study. The total cost per PSW for training and supervision was £2548, or £292 per patient. Patients had an average of 6 appointments (95% CI 5.5 to 6.5 appointments), with a total cost of the peer-support intervention of £132 (95% CI £118 to £145). Including mental health acute care and community costs calculated from patient files, the total cost over 12 months adjusting for baseline was £6459 (95% CI £4766 to £8151) for the peer-facilitated self-management intervention and £6720 (95% CI £5329 to £8112) for the control group, with a mean incremental cost difference of –£261 (95% CI –£2450 to £1928). Patients in the peer-facilitated self-management group had an average of 0.991 (95% CI 0.931 to 1.051) QALYs over 18 months, baseline adjusted, compared with 0.968 (95% CI 0.907 to 1.03) QALYs for the treatment-as-usual group, with mean incremental QALYs per patient of 0.023 (95% CI –0.062 to 0.107). Drawing a straight line between 4- and 18-month QALYs, the mean QALYs at 12 months are 0.651 (95% CI 0.612 to 0.689) for the PSW group and 0.640 (95% CI 0.600 to 0.679) for the workbook-only group, with a mean difference of 0.011 (95% CI –0.043 to 0.065). The intervention dominates treatment as usual (more QALYs for a lower cost), with a 64% probability that the intervention is cost-effective compared with treatment as usual at 12 months and at £20,000 willingness to pay for a QALY gained. For a full report of the health economic analysis, see Appendix 13. This was a rigorously conducted RCT with blinding of researchers and data analysts, although blinding of participants was not possible. Data for the primary outcome were collected from patient records from blinded NHS staff, although clinicians working with the participants could not be blinded. A power calculation determined sample size. As anticipated when planning the project, an excellent follow-up rate was obtained for the primary outcome when data were collected from patient records, and good follow-up rates of nearly 80% were obtained at 4 months, although they fell below 60% at 18 months. There is a 64% probability that the intervention is cost-effective compared with treatment as usual, resulting in more QALYs for lower costs, even after the cost of the intervention was taken into account. The difference in costs and QALYs is not statistically significant though, with wide CIs for the costs. This is the main reason that the probability that the intervention is cost-effective is lower than expected, given that the intervention dominates treatment as usual. One limitation of the study could arise from the way participants were recruited. CRT staff identified potentially eligible service users from their caseload; therefore, it was possible that those recruited were service users who were well engaged with, and easy to contact by, CRT staff. Patient data for five participants at baseline and five participants at follow-up could not be retrieved by NHS IT teams and, thus, had to be treated as missing. A completion rate of > 60% was not achieved for the 18-month follow-up interview because of difficulty in contacting and consenting participants. This was despite efforts to maximise the response rate by offering participants shorter telephone interviews as an alternative to face-to-face interviews. The 18-month follow-up was planned once participant recruitment had been completed, with the agreement and additional funds from the study funder. Another limitation with potential to introduce bias and limit generalisability is that only 24% of those eligible were recruited to the trial. Research is required to further investigate the critical ingredients and underlying mechanisms of change of this successful intervention. A planned process analysis using data from this study can help with this, exploring the relationship with outcomes of completion of the recovery plan, therapeutic alliance with the PSW and the PSW’s perceived recovery orientation, and the degree of match between PSW and service users on demographic and clinical variables. Replication studies to corroborate the findings from this trial are desirable. However, the intervention appears successful as a means of reducing relapse in people following discharge from a CRT after home treatment, a result that coheres with the success of other self-management interventions in people with mental health problems.95,121 The reduction in relapse also translates to a potential reduction in the average cost per patient, compensating the cost of the intervention. There is thus a good case for wider dissemination of such interventions in routine NHS settings, and for research to investigate the best methods of supporting their implementation. Development work for workstream 2 confirmed the acceptability to people using CRTs of additional post-crisis support from a PSW, and of support with relapse prevention. Given the findings from workstream 1, about how infrequently relapse prevention work is provided as part of CRT support,63 such support can address an unmet need for CRT service users. Our preliminary testing established that it is feasible for PSWs to deliver a structured self-management programme in this post-crisis context. An established framework for developing complex interventions was followed:100 evidence review, stakeholder consultation, modelling and feasibility testing, then piloting. This ensured that before evaluation in a trial, the peer-supported, self-management programme and the operating procedures for delivering it had been developed and refined to maximise utility, acceptability and feasibility. The development time, funded as part of the CORE programme, therefore gave the intervention the best chance of demonstrating effectiveness. The primary hypothesis was confirmed, suggesting that this intervention is a potentially useful one in crisis populations in mental health service settings. The result persisted with adjustment for whether or not participants were in a SMI psychosis/bipolar disorder group, suggesting that the intervention may be helpful for people with a wide range of mental health conditions following CRT support. The absolute reduction in admissions in the trial (i.e. 29% in the intervention group compared with 38% in the control group) was smaller than the 15% reduction that was anticipated when planning the trial. However, rates of re-admission to acute care were lower than originally anticipated in both groups. (The sample size calculation was based on a reduction in re-admissions from 50% to 35% of participants.) These relatively low admission rates to acute care could be explained by the most unwell CRT service users not being recruited to the trial, or it may be that using the recovery plan without support from a PSW had some effect in reducing admissions for control group participants. Therefore, the absolute risk difference between groups was understandably smaller than anticipated (i.e. 9% rather than 15%), but the relative risk difference between groups in the actual results was broadly similar to what had originally been anticipated. Given the importance of acute admission as an outcome, the intervention effect is considered, with a number needed to treat of about 11 to avoid a re-admission, to be potentially meaningful as well as statistically significant, although the limited impact on secondary outcomes at each time point is also noted. This finding should be seen in the context of an already promising evidence base for self-management interventions for SMI (see review in Appendix 8).95 Given these prior positive findings, it is suggested that the evidence base for the clinical effectiveness of self-management interventions for SMI is now notably strong. However, although the study has found an effect on relapse that had not clearly been demonstrated previously, findings on some secondary outcomes were disappointing in comparison with other self-management studies, in which effects have been found on measures such as symptoms, hope and empowerment. This makes mechanisms of effect relatively difficult to explain in the study. The secondary measures tended to be summary scores encompassing a large number of domains; it may be that more specific measures of potential intervention targets would have shown more effect. The findings from the workstream 2 trial substantially augment the evidence base for peer-support interventions for SMI, providing the most robust evidence yet for the clinical effectiveness and cost-effectiveness of any peer-provided intervention in a UK secondary mental health setting. It adds to evidence from recent US trials that structured, peer-delivered recovery-focused interventions, including a relapse prevention component – such as a wellness and recovery action plan121 or recovery workbook48 – can improve outcomes for mental health service users. It helps address the evidence gap for peer support in mental health, whereby peer support is increasingly commonly provided in mental health settings97 in the absence previously of strong evidence for its clinical effectiveness and cost-effectiveness.103 The design and procedures for the workstream 2 trial were robust, with oversight from a clinical trials unit regarding trial procedures and management. The primary outcome of re-admission to acute care was objectively measured, and outcome data were provided for 98% of participants. This can give a high level of confidence about the validity of the positive trial finding. An acceptable response rate of nearly 80% was also achieved for the end-of-treatment secondary outcomes. The lower response rate for secondary outcomes at the 18-month follow-up of just below 60% is a limitation of the study. The randomised trial context in which the intervention was delivered had some impact on how it was implemented and how trial results may be reflected in routine care. About one in seven CRT service users screened and one in four of those eligible took part in the trial. However, it is unclear to what extent this reflects the proportion of CRT service users who might accept this intervention if offered as part of routine care. Take-up rate may have been affected by the need to consent to participate in a trial. Not all eligible service users could be contacted to ask about interest in the programme; more might have been contacted by involved clinicians during CRT support if the intervention were part of routine care. Peer-support workers delivering the intervention were only employed to work with trial participants, so the timing and amount of work on offer were not wholly predictable. PSWs at most sites were therefore employed on zero-hours contracts and at times faced challenges to fit meetings with participants around other commitments. Many PSWs were delivering an intervention of exactly this nature for the first time in the trial; their confidence and familiarity with the recovery plan are likely to have improved during the course of the trial. Integration and lines of communication with CRT teams and supervisors also needed to be developed during the course of the trial. The extent of any negative impact of these factors on the quality of the study intervention delivered or its effectiveness are unclear; however, these difficulties are likely to be mitigated if the intervention was embedded as a routine part of CRT services. There are therefore some grounds for optimism that the intervention could be more effective in routine care than in this trial context. The positive trial result generates a number of fruitful avenues for future research. First, replication of the findings from this trial in future studies is desirable. Second, a better understanding of the critical components and mechanisms of effect of the intervention is needed. This is particularly pertinent, given the negative results for secondary outcomes that may have been expected to result in reduced rates of relapse (reduced symptoms, better personal recovery, more social support). A planned process evaluation from the CORE trial data will attempt to identify whether or not there are independent effects on outcomes from the relationship with the PSW and completion of the recovery plan. Adaptations to the study intervention are also of interest in understanding its critical components. Whether the self-management plan support could be provided equally well by non-PSWs or whether other forms of peer support might be as effective in supporting people post crisis are potential directions for future research. Third, there is a need to understand why some CRT users declined the intervention or discontinued it (i.e. nearly 30% of those participants in the intervention group), and how their needs might best be addressed. The qualitative interviews with participants in the piloting phases of our study indicated that a minority of service users were ambivalent or unenthusiastic about making a written recovery plan, but the extent to which this dissuaded people from taking up the intervention or continuing with it is unclear. Future qualitative studies could helpfully explore this. Last, the utility of the trial intervention in other mental health service settings is of interest, for instance for service users following other types of acute care, or in community services providing longer-term support. The main clinical implication of this work is that our findings support the wider use of the CORE peer-provided self-management programme as a means to reduce relapse and increase satisfaction with care for people discharged from CRT services. As well as PPI contributions to overall study design and planning, the CORE study included PPI involvement through members of the study working groups in the following activities. Topic guides for qualitative interviews with service users and carers in module 1 (CRT stakeholder interviews) and modules 4 and 5 (peer-supported self-management programme; interviews with participants in preliminary testing and piloting) were developed in collaboration with the study working group. A draft version of the information sheet for the workstream 2 trial was also reviewed by peer researchers through the MHRN’s Feasibility And Support to Timely recruitment for Research (FAST-R) reviewing service.122 Twelve working group members, the public involvement co-ordinator and two research assistants with relevant lived experience conducted interviews in module 1 (CRT stakeholder interviews). This ensured that 54 of 61 interviews (89%) with service user and carer participants were conducted by peer interviewers. The peers from the working groups were being paid £60 per interview. Working group interviewers were provided with 2 days’ training in interviewing skills. Support was also given through three group supervision meetings and feedback from the programme manager who listened to interviews. The CRT stakeholder interviews were analysed using thematic analysis. An initial coding frame was drawn up by the programme manager. Fifteen of the peer researchers (including 11 who did interviews) then reviewed at least one transcript and gave advice on any necessary revisions to the coding frame (in addition to similar input from other members of the study team). A 2-hour basic training session was provided to help with this task, and there was also a follow-up reflection meeting. An opportunity was also subsequently given to advise on a draft write-up of a paper relating to the analysis. Seven focus groups were conducted with service users (n = 5) and carers (n = 2) to seek feedback on initial plans for a peer-supported self-management programme for use in workstream 2. Six of these focus groups were facilitated by peer interviewers. In addition, a black and minority ethnic (BME) focus group about views on CRT care was convened and conducted by a member of the working group to address a concern that BME service users’ views may have been under-represented in CRT stakeholder interviews in module 1. Peer researchers were involved throughout the development of the fidelity measure. Two research assistants with lived experience contributed to reducing a longlist of 330 statements about CRTs down to 72 statements. These 72 statements were grouped and prioritised by stakeholders in a concept mapping process, which involved 16 service users and six carers, and generated the 39-item scale. Peers from the working groups had opportunities to give input to eight iterations of the fidelity measure as it was refined. Peer researchers also inputted into the development of the questions for service users and carers in the fidelity reviews. Six out of 18 of those taking part in the inter-rater reliability testing exercise for the fidelity scale were peers from the study working group. Nine of the working group members, the public involvement co-ordinator and five research assistants with lived experience acted as reviewers in the CRT fidelity reviews on the study. All 140 reviews (75 in the module 2 CRT survey, 65 during the module 3 trial) were thus able to include a service user or carer reviewer. Half a day’s training, plus e-training, was provided to reviewers. The content of the online CRT resource kit was discussed in two working group meetings and members contributed material relevant to enhancing users’ experience in CRTs. Four members of the working group and the public involvement co-ordinator provided video testimonials that were included in the resource kit of the SIP. The working group provided advice on refining workbook content and procedures for the peer-support programme. In addition, all four peers who delivered the peer support at the pre-pilot stage (module 4) and three of five at pilot stage (module 5) were from the CORE working group (including the public involvement co-ordinator). This enabled further refinement of the intervention processes and workbook content, including through a focus group. In the main peer-supported self-management programme trial (module 6), training for the PSWs in all six study sites was co-facilitated by a peer trainer, one of three members of the CORE working group who had worked as a PSW in the piloting stages. A 5-day training course was provided (reduced for smaller groups of trainees). Two ‘top-up’ training meetings were also provided for all PSWs on the trial, which were co-facilitated by two PSWs from the working group. The public involvement co-ordinator also visited four of the sites where no peer supervisor was available to provide peer supervision, and was available for added support if wanted. Three members of the study working group presented on study PPI and the peer-support project at two study symposium events. The study public involvement co-ordinator has contributed to a number of study publications as a co-author. The study working group members are acknowledged on three published and planned papers in which they have had significant involvement regarding the development of the CRT fidelity scale, CRT stakeholders’ views on CRTs and the development of the peer-supported, self-management programme. Following publication of the workstream 1 and 2 trial results, there is an opportunity for working group members to lead on writing a blog or report on each trial aimed at service user and carer readers. Other public involvement on the study has included (1) interview panels for researcher posts have included a peer interviewer approximately 50% of the time, (2) 12 members of the working group contributed to informal interviews about their experience and reflections on PPI in the study and (3) a research assistant with relevant lived experience has led on conducting qualitative interviews with PSW teams (n = 6) as an addition to the funded study. Working group members’ and researchers’ reflections on the experience and impact of PPI on the CORE study is reported in the following section. Advice from the study working groups contributed to all aspects of study planning. It is not always possible to unpick which or to what extent decisions were influenced by working group involvement. However, five examples of decisions about study plans that originated from the working group and were adopted directly following PPI advice are as follows:
An independent consultant (SP) collaborated with a study co-applicant (TW) to explore user experiences of involvement in the programme. The following brief report outlines this. Authors: Sue Patterson, Principal Research Fellow – Mental Health, Metro North Hospital and Health Service (MNHHS), QLD, Australia and Associate Professor, Department of Applied Psychology, Griffith University, Mount Gravatt, QLD, Australia; and Tim Weaver, Associate Professor of Mental Health Services Research, Middlesex University, London, UK. Policies internationally endorse the active participation of non-professional stakeholders in the design and delivery of health and social care. The UK policy commitment to PPI in research is supported by investment in infrastructure designed to enable involvement. Funders, governance and support bodies commonly require evidence that people potentially affected by the research are involved in various ways.123,124 However, it remains for researchers to determine the purpose and methods of involvement. Consequently, the involvement of service users has been enacted in diverse ways at different stages of the research process. Researchers have been called on to examine user involvement in studies they lead.125 This report represents one such examination. Service users and carers have been variously involved in the CORE programme:
The CORE programme management group sought to examine the clinical ‘effectiveness’ and acceptability of the PPI model employed in the CORE programme, with reference to experience of involvement and views about what worked well and areas for improvement. The aims of the study were therefore to:
A qualitative approach was adopted and was underpinned by an interpretive social paradigm.126 This qualitative study received ethics approval from the Middlesex University Health and Education Ethics Committee on 22 June 2015 (reference number MH19). Participation in the evaluation was open to all people involved in CORE programme PPI activities. Potential participants were approached by a CORE programme manager seeking permission to provide their contact details to Sue Patterson. Contributors were provided with an information sheet and (if they agreed to an interview) this interview was conducted at the participant’s convenience. Interviews were conducted by Sue Patterson during July and August 2015. Interviews were semistructured, audio-recorded and transcribed verbatim. Participants received a £20 honorarium. Analysis involved developing summary accounts of each participant’s views, coding and labelling content in accordance with study objectives. An initial analysis was presented to the PPI working group and feedback informed further analysis and interpretation. Interviews were completed with 12 contributors aged between 30 and 76 years. Most were women (n = 9). Although contributors had in common ‘lived experiences’ with mental illness and mental health services, contributors reported varied life histories. The majority reported substantial experience of PPI. Involvement in the CORE programme covered the activities listed in Introduction but varied in duration. Some (n = 7) reported continuous involvement from ‘the beginning’, and a minority had become involved subsequently (n = 2) or withdrew (n = 3). Contributors shared the view that PPI in research generally aimed to improve processes and outcomes by ‘grounding’ research, enhancing its ethical status and improving feasibility, acceptability, relevance and credibility. Contributors commonly linked their involvement in the CORE programme’s PPI to personal well-being, the intrinsic rewards associated with making a contribution and an aspiration to improve mental health services. The study found broad consensus that the ‘process’ of involvement was professionally, respectfully and efficiently managed. Continuity of management throughout was considered particularly important in this regard. Most contributors reported that opportunities for involvement had been clearly explained and often made favourable comparisons with other involvement experiences. A minority, however, reported an initial uncertainty about expectations and expressed concern about lack of communication between working group meetings. The CORE programme team was commended by interview respondents for the management of payments to enable participation of people receiving income support. Contributors reported that their contact was primarily with the programme manager and PPI co-ordinator. The consensus was that the research team collectively and individually were accepting, friendly, caring and approachable. Many contributors described instances when they felt that they had been treated sensitively and respectfully. Most contributors thought that their input was valued. Some contributors mentioned other researchers on the team but said that they did not know the chief investigator or understand the role of the service user co-applicant. Working groups were described by the majority of collaborators as cohesive and functional. Generally, contributors considered the discussion at working groups on topic, appropriate and constructive. Although reporting that debate was sometimes robust, most contributors considered this appropriate and respectful. Many spoke of ‘learning’ about and deepening understanding of the research and workings of CRTs. Divergent opinions were expressed by a minority, with concern expressed about:
Contributors found the development of the CRT fidelity measure intellectually and philosophically challenging. Although some contributors said that they understood the methodology and the process, others reported confusion. The process was described as particularly difficult, with some contributors struggling to prioritise the elements of a ‘good’ crisis service. Concern was expressed by some that their views in relation to the appropriateness or number of items were not given sufficient weight. Contributors who had conducted qualitative interviews typically viewed this as appropriate and reported enjoying and learning from the process. While acknowledging that assumptions of shared understanding or language could be problematic, the contributors spoke of observing interview participants relaxing when their status as a ‘peer’ was disclosed; most expressed the view that peer interviewers generated better (‘more honest’) data. Some contributors, however, reported that hearing others recount distressing experiences could ‘be close to home’ and that they struggled to maintain boundaries and the interviewer role at times. Few of the interviewed contributors reported involvement in analysis of qualitative data. Although this limits our ability to draw conclusions about participants’ experience of this aspect, those who were involved in analysis did report some discomfort with the task and uncertainty about how feedback provided was incorporated into the final analysis. Contributors reported participating in up to 20 fidelity reviews. Some endorsed the review process enthusiastically and considered their input valued both by the service being reviewed and the research team. Several reported difficulties adhering to the interview schedule and wanted to explore issues that seemed important to themselves or the person being interviewed. Some contibutors were concerned about the use of a standardised measure in vastly different contexts and questioned the appropriateness of evaluating quality quantitatively. Most contributors welcomed the opportunity to express views and, although wanting these heard, were aware and accepting of the constraints within which a programme of research, such as CORE, operated. Some contributors, however, felt constrained, noting that because some working group meetings were convened after the ethics review of the relevant CORE substudy was complete, the scope for amending study procedures was limited by the willingness of the investigators to submit amendments to the Research Ethics Committee (and tolerate any consequent delays). Although noting that impact was difficult to assess, contributors were typically confident that PPI had shaped or influenced the process of the CORE programme in various ways:
The majority of contributors described their participation as developing interpersonal and practical skills they considered valuable. Many spoke of ‘learning’ about research or the ways health services ‘work’. Contributors commonly spoke of being affected emotionally or psychologically. Sometimes experiences were transient (e.g. ‘being upset’ listening to service users’ accounts), sometimes lasting – contributors spoke of deepening sense of self-esteem and respect, of coming to develop a new (and preferred) sense of self. Some contributors reported that involvement with the CORE programme had facilitated further involvement opportunities. The majority of contributors described PPI in the CORE programme as robust, meaningful and worthwhile, and generally experienced it as a rewarding and intellectually stimulating activity. Contributors linked the maintenance of their involvement to feeling respected and trusting that their input was valued. However, some (including two who discontinued participation) expressed the view that they and their input were marginalised and reported dissatisfaction with the process and the level of involvement achieved in relation to aspects of research design. Several factors require caution in interpreting these findings. The timing of the interviews some 4 years into the CORE programme may have affected recall. Moreover, not all PPI contributors participated, and it is entirely possible that those who did not take part in interviews had different experiences and views. What worked well?
What could be improved?
The evaluation was undertaken by Dr Sue Patterson as part of the programme of work supported by the MNHHS Chairman’s Scholarship award with in-kind support of the MNHHS. Dr Tim Weaver is a co-investigator on the original CORE programme funding application. His involvement has been advisory, and he had no formal responsibility or involvement in the major workstreams to which the service user and carer working groups contributed. No payment was made for the evaluation. Expenses for Dr Patterson to travel to interviews, transcription costs and honoraria for participants were paid from the CORE programme research project budget. Thanks go to members of the CORE programme team for their administrative and logistical support. Sincere thanks go to the PPI contributors and research team members who generously shared their views. The national implementation of CRTs in England was proposed as a means of resolving the ‘acute care crisis’,127 and trial evidence indeed suggests that CRTs can reduce the number of inpatient admissions and increase satisfaction with acute care.18 However, it is less clear if all the aspirations of stakeholders for CRTs are met by CRTs as implemented in practice in England. When scaled up to a national level, CRTs’ impact on reducing bed use and costs is uncertain.13,14 Rates of re-admission to acute care following support from a CRT may be as high as 50% over 1 year.91 A number of recent reports by government and expert bodies have been highly critical of CRTs and acute care generally.28,85,86,88 A recent report by the Care Quality Commission88 found that only 14% of service users felt that they had received the right care from mental health services during a mental health crisis. Recurrently identified problems include a lack of continuity of care and responsiveness, an unduly limited range of interventions, fragmented therapeutic relationships and a lack of support at (and following) the point of discharge.58 In this context, the twin aims of the CORE study – of developing evidence about how to implement high-quality CRT services and reduce relapse for people who have used CRT services – are of high importance for policy-makers and people who work in or with CRTs or who use CRT services. The development work for workstream 1 established the following key points. First, the model of CRT care endorsed in The NHS Plan1 and accompanying policy guidance3 when CRTs were mandated nationally retains broad support. An easily accessible CRT service that provides rapid-response, brief, intensive support to people who would otherwise be admitted to hospital, which offers medical psychological and social support to service users, and involves their families and other social supporters was endorsed across all stakeholder groups with a high degree of consensus. Second, this model of CRT care is feasible: the national fidelity review conducted for the CORE study found that all aspects of a specified good practice model for CRTs were achieved by some teams. Third, CRT implementation nationally is highly variable;63 CRTs are working in varying acute and community service contexts, and few teams are achieving consistently high-quality care. A national mandate and policy directives have proved insufficient to achieve consistent implementation as planned among CRTs nationally. Two outputs from the CORE study, the CRT fidelity scale69 and the CRT SIP,73 sought to meet this need for more successful implementation of a CRT best practice model. The CRT fidelity scale encapsulates a highly specified model of CRT good practice and provides a means to assess how far CRT teams are achieving such good practice. Its thorough development process, including a large stakeholder concept mapping exercise, ensures that it reflects the best available evidence and the priorities of all stakeholder groups for CRTs. There was a good level of consensus between stakeholder groups, including service users and carers and clinicians, on the key ingredients of high-quality CRT care. An encouraging conclusion from the study is that the CRT fidelity review audit process developed in the CORE study was experienced as helpful by teams. Seventy-five CRTs were recruited as planned for the study’s fidelity survey; all 75 CRTs were retained in the CORE trial, which involved, for many teams, four reviews over a 2-year period. Qualitative feedback from CRT teams and managers about the fidelity scale was positive. Promising evidence for the reliability of scores from fidelity reviews has been established. However, the limited extent of associations between total fidelity scores and outcomes established in the CORE trial indicates that there is further work needed to confirm or improve the psychometric robustness of the CRT fidelity scale, or to identify the most critical components of service quality within it or which components are most associated with which aspects of CRT model fidelity. The CRT SIP was also well received by teams. The national fidelity survey found that access to supervision, mentoring or CRT-specific training is very limited for many CRT staff. The SIP’s team scoping day to plan service priorities, receive support from an experienced CRT facilitator and enable access to practical resources to support good practice offered welcome resources to equip teams to deliver service improvement. The CORE SIP trial results suggest that it did not meet all its aims. In particular, the study found no significant difference in service user satisfaction between teams receiving the SIP and controls (although it is possible that the lack of a positive finding was because of the unrepresentative, highly satisfied service user participant group recruited to the trial). Promisingly, however, the SIP did prove able to increase teams’ fidelity scores, and there was evidence of reductions in admission rates and inpatient bed use compared with a control group of teams not receiving this. These promising results suggest that the SIP has value in itself to support service improvements in teams, and provides a useful resource from which to develop and evaluate refined implementation resources in future. The barriers to full take up of the SIP in teams, identified by the study’s qualitative evaluation, can help direct future efforts. In particular, the online resource pack appears to have been underused, and ensuring staff time for SIP activities and buy-ins from senior management was challenging. Current plans for more assertive use of policy levers to monitor and promote quality improvement in CRTs in line with CORE fidelity specifications may also be of value to focus service improvement efforts and achieve management and commissioners’ active support. The Achieving Better Access Programme128 exemplifies these attempts: it provides a 5-year programme to clarify quality standards and audit delivery of these more rigorously, in order to improve access to services and evidence-based treatments within them across mental health care. The CORE fidelity standards have fed into the Achieving Better Access Programme standards128 for mental health acute care. Workstream 2 of the CORE programme focused on providing effective post-crisis support, aiming to ameliorate the often-abrupt transition from intensive CRT support to follow-on care45 and to reduce the known high rates of relapse (as measured by re-admission to acute care)91 following CRT service use. Thus, reduction in relapse (as measured by re-admission to acute care) was selected as the primary aim of the intervention and the primary trial outcome, being highly relevant to both service users’ experience and recovery and service costs. Development work for workstream 2 established the promising evidence base for self-management interventions in mental health95 and the acceptability, in principle, to CRT users of peer-provided support focusing on relapse prevention. It allowed a careful process of developing, testing and refining a peer-provided self-management intervention for testing in a trial. The study was greatly helped by being able to draw on existing high-quality resources. First, the study produced an adapted version of the Personal Recovery Plan,129 a mental health self-management resource developed by Rachel Perkins and Julie Repper in co-production with service users. Second, the training team from the Nottingham Institute for Mental Health helped the study team develop and deliver a bespoke, adapted and abbreviated version of its accredited peer-support training for the PSWs delivering the trial intervention. A pre-pilot phase allowed refinement of the intervention content and procedures before the main trial. The benefits of this staged, development and piloting process, drawing on best available resources and evidence as recommended in the MRC guidelines for evaluating complex interventions,100 are confirmed by the positive trial results. The CORE trial of a peer-provided, self-management programme for people leaving CRT services produced a positive result on its primary outcome in a robust RCT: rates of re-admission to acute care were significantly lower for those in the intervention group than in the control group (29% vs. 38%). The positive result for satisfaction with care suggests that the intervention enhanced service users’ experience of care too. Not all the trial outcomes had positive results. The intervention appears to have no effect on residual symptom levels or social outcomes (i.e. employment, social network, loneliness); different types of support may be required to address these needs. No benefits were apparent on secondary outcomes at the 18-month follow-up: the effects of the intervention may dissipate over time. Nevertheless, the positive outcomes that were achieved were of high importance to service users’ experience, clinical recovery and service costs, and constitute an effective intervention. This positive trial result generates further questions of interest. It is not known, for example, whether or not other types of service provider could deliver the self-management programme as effectively as the PSWs in this trial. It is equally unknown whether or not other types of peer support (e.g. less structured programmes) for people following a mental health crisis could produce similarly positive results. In the light of the trial’s positive findings, these are fruitful lines of research. A current trial of a peer-support programme for people leaving acute inpatient care68 will provide a useful addition to the developing evidence base regarding peer-provided support in NHS acute settings. Two conclusions can be drawn from the CORE workstream 2 trial. First, it adds to the promising evidence base95 for self-management interventions in mental health as a means to promote recovery and reduce relapse. The CORE fidelity survey69 highlighted that recovery plans or collaborative relapse prevention plans are seldom offered routinely in CRTs. Our study illustrates a mechanism through which this type of support can be provided effectively to people following a period of CRT support. Further research about optimal means and settings in which to deliver self-management interventions in mental health is a high priority given the low costs and high feasibility of implementing self-management programmes compared with many other types of support. Second, the trial adds significantly to the current evidence regarding peer support for SMI (i.e. people using secondary mental health services, including a substantial proportion with psychosis and bipolar disorder) in particular. The employment of PSWs has proliferated in the NHS in the last decade,98 ahead of a corresponding evidence base. Although there is some existing evidence that peer support can help reduce symptoms for people with depression,130 the evidence regarding peer support for people with SMI is less robust.103 The trial found reduced levels of re-admission to acute care (29% in the intervention group compared with 38% in the control group: a reduction that is statistically significant and is believed to be clinically meaningful) for a participant group including people with common mental disorders and almost 50% of participants with bipolar disorder or psychosis. The primary trial outcome remained statistically significant even with adjustment for whether or not participants were in this bipolar/psychosis group. Based on two recent systematic reviews,103,131 it is believed that this study is the first UK multisite randomised trial of a peer-provided intervention for people with SMI, and the first trial anywhere to demonstrate an objectively measured reduction in acute service use for this group in a peer-provided intervention. Overall, the CORE programme has produced a set of resources that help to specify and assess optimal CRT care, and can help CRT teams with achieving quality improvement. It has also trialled an effective peer-provided self-management programme to prevent relapse for people following CRT care. These outputs can help achieve the overarching goal of improving care for people experiencing a mental health crisis. Summary of main research recommendations
|
What are the four steps that a crt follows when engaging in primary prevention?
Pos Terkait
Copyright © 2024 membukakan Inc.
