Hurdles and potentials when implementing internet-delivered Acceptance and commitment therapy for chronic pain: a retrospective appraisal using the Quality implementation framework

Implementation outcomes

This study retrospectively described the actions taken during an initial implementation initiative, in the purpose to develop a logic model to guide future implementations. The implementation outcome in focus was primarily appropriateness. Appropriateness refers to the perceived compatibility of a treatment to a setting [42], in this study the compatibility of IACT for chronic pain as an add on treatment to IPRP at specialist level. Different facets of appropriateness were of interest, such as suitability, usefulness and practicability. Also, certain aspects of adoption were of interest, such as utilization and intention to try. Adoption refers to the intention to employ an evidence-based practice [42], in this study implementation actions that showed the organization’s interest in adopting IACT as addition to IPRP. Lastly, actions relating to sustainability were found in the data as it was compared to the emerging literature on implementation of internet-delivered interventions in regular health care. Sustainability refers to the maintenance of a novel intervention [42], in this study actions that have potential to aid the incorporation of IACT in an IPRP setting.

Design

Hybrid 1 trials are recommended when there is strong evidence to support the applicability of an intervention in a setting, although more knowledge of the implementation process is needed to address implementation research questions more accurately [48]. Valuable information may then be gathered in the purpose to better the fit of the intervention and the implementation logic for subsequent trials. Collected data should target facilitators, barriers, translation problems, promising implementation strategies and potential modification. In this hybrid 1 effectiveness-implementation study [48], IACT was implemented in a specialty care setting. In addition to evaluating the effects of the intervention which has been published [15], data from the implementation process were evaluated retrospectively using a deductive approach in line with the Quality Implementation Framework (QIF) [49].

In a hybrid 1 trial [48] the primary goal is to test the effectiveness of an intervention. The secondary aim is to gain knowledge of contextual factors that may influence the implementation process to learn if a particular intervention is suitable in a certain context or for a certain patient group. Even though determining the effectiveness of the intervention is in focus, data on the implementation process is still collected as the trial runs. However, the data is usually not summarized and analyzed until the end of the trial [50, 51]. One challenge with this process is memory loss in cases of interviews, or failure to realize what data should be stored for later processing. However, an advantage with retrospective appraisals is that data from the start of the implementation process to the end may be evaluated taken together. Also, it is more likely that the effect of strategies from the earliest phases of the implementation process becomes apparent, if the appraisal covers the complete implementation time, compared to evaluating the process as it occurs.

Implementation model

Of the eligible TMF:s, QIF [49] was chosen, primarily because of its applicability and usability for describing actions taken when implementing a novel intervention in a fixed context. Evaluated by T-Cast – a tool to compare and select a suitable TMF – [52] the QIF was found to address the relevant analytical levels (individual consumer, individual provider, care team, organization), provide a sequential approach and cover important constructs. The QIF also includes procedures for practical implementation. Most importantly, it illustrates how actions in different steps of implementation influence each other, for example how the initial analyses affect decision making processes later on. The QIF has previously been applied to detect potentials and hurdles for sustained implementation [53]. It has also been helpful in retrospective evaluations when a novel interventions was implemented in a specific setting [54].

QIF is the result of an extensive merge of 25 implementation frameworks from several research areas, focusing on common crucial steps for implementation [49]. QIF is structured in four sequential phases and implies which practical actions need to be taken in each step. It has so far been applied in diverse settings, both when planning implementation [55] and evaluating an implementation process [51]. The first phase of QIF concerns initial considerations regarding the host setting and conveys eight strategies for assessment, adaption, and capacity-building. The second phase includes two strategies to create a structure for implementation. The third phase contains three strategies for ongoing structure once implementation begins. The final forth phase focuses on improving future initiatives. However, the evolvement and adaption of actions may continue throughout the process and are not bound to its respective phase. Also, contextual factors may influence how and when in the process the steps are practiced and emphasized.

Implementation setting

In a specialist level clinic, chronic pain patients were randomized to a 6-week IPRP either with or without the addition of IACT. Results showed medium to large between group effect sizes on pain acceptance, psychological inflexibility, affective distress, and self-efficacy for the group who received IACT in addition to IPRP. However, large missing data, high dropout rates and heterogenous data complicated interpretation of the results [15]. It was concluded that IACT may enhance the effect of IPRP on psychological outcomes, although future studies on contextual features of IPRP were suggested to shed light on implementation hurdles as attrition.

Implementation object

The implemented object was a web site providing psychological interventions, information on rehabilitation and means to communicate with an e-therapist. The platform behind the website was designed and developed by two software engineers. Two psychologists with previous experience of ICBT and IACT [38, 56, 57] wrote the specification. For technical details on code, storage, and database, please see Supplementary material.

Intervention

The content on the web page was updated weekly and built on to what patients were presented during a 6 week long IPRP on cite. The material was altogether ACT based, including audio recorded mindfulness and experiential exercises, along with educational texts on chronic pain and self-management. The web site also presented an exercise diary, forms for evaluating behavior change and audio recordings of physiotherapy interventions. The 11 week long aftercare part, given after the end of the IPRP, continued with psychoeducational texts, means to continuously plan and evaluate behavior change along with mindfulness and experiential exercises. Congruent with content earlier presented during the IPRP, the web page also contained insomnia interventions and texts with exercises based on occupational therapy and physiotherapy. The rationale for the design was to provide opportunities for patients to practice rehabilitation activities in between sessions and after the end of the IPRP, in their home-setting. The aim of the RCT was to study the effectiveness of IACT when added during and after an IPRP for chronic pain patients in specialist care [15].

Analysis

All eligible project documents, digital or hand-written, were collected by the first author in collaboration with the project team members. The documents consisted of reports, applications, communication with stakeholders, in-house evaluations, planning document and notes taken at team and unit meetings (see footnotes in Table 1).

All documents were initially scanned to assess whether they conveyed information on QIF implementation strategies, see Figure 1. All official documents, as reports and applications were deemed informative. Following discussions in the research team most of the hand-written notes were not considered informative but are kept for potential later analyses. The next step was a more thorough review of the remaining documents with one QIF-phase in mind at a time, in consecutive order. Information explicitly or inexplicitly described, or not at all found, based on the fourteen QIF-steps were summarized in a table during the read-through, see Table 1.

Figure 1:

Flow chart of analysis.

When all documents had been screened for data concerning all 14 QIF-steps, the data was summarized for each of the four phases; (1) initial consideration, (2) creating a structure, (3) ongoing strategies, (4) learning, see the Results section. This summary describes which implementation actions that were or were not described in the documents.

Thereafter, patterns of behaviors, reasons for decisions made and possible explanations to actions taken or non-taken were checked. Differences in actions during the four phases were sought for. Also, links between actions in the four QIF-phases were looked for from a time-perspective, to see if early actions affected later ones. This evaluation resulted in grouping of data into two categories; implementation actions that had either facilitated of challenged the implementation process, see Figure 2. Lastly, conclusions were summarized in a logic model to guide future initiatives, see Figure 3. The consistency of findings was then checked by the research team. An inquiry audit was made by a researcher with comprehensive expertise in pain rehabilitation although not explicitly involved in the analysis process, to evaluate the consistency of findings.

Figure 2:

Facilitators and challenges in the implementation initiative, summarized in each of the four phases of QIF respectively.

Figure 3:

Logic model of implementation of IACT added before or after IPRP at a specialist pain clinic.

Phase 1: initial considerations regarding the host setting

Phase 1 actions were described primarily in funding applications and ethical approval protocols. Although several important facilitators were found, as leadership buy-in, this phase contains the most challenges. Noteworthy is that assumptions on influential factors, appears to build on experiences from previous implementation projects of the same intervention although for different patient groups or in other settings. Applying learned experiences onto the present host setting was probably accurate to a certain degree. However, patient-specific obstacles might have needed more attention, and this may partly explain why some implementation actions were not prioritized. First, the actions taken in steps 1 and 2 imply that the intervention was assumed to match the organization’s vision and there through also fit problems that were expected to happen within a near future (i.e., increased number of referred patients and stake-holder expectancies to transit to digital care). Secondly, benefits for staff, present organizational needs and consumer cultural preferences were not explicitly outlined at this stage. Due to short deadlines, the modifications made were tested by software engineers and staff, but not by end-users. Consequently, host setting feedback was not collected. Also, the absence of a plan for adoptions might be one reason to why changes occurring throughout the implementation process were not monitored or documented in a structured way.

The confidence in the intervention was high. Recruited staff had experience from previous successful implementation initiatives. Several of the competencies and roles listed in the QIF’s crucial steps were indeed considered during team recruitment and capacity building. In some cases, the same person had several roles, as is common and often inevitable due to limited resources. However, required competences were sometimes implicit. Although supervision was frequent in the latter phases of implementation, the team worked more independently in the earliest stage. Guidance from supervisors put the light on barriers that later would become apparent. However, some challenges were not foreseen, possibly due to hasty assessments in the initial phase and un-attentiveness to patient-specific barriers. Opportunities for staff feedback existed. A dedicated controller could have monitored this process. Additional host staff-recruited champions could have eased implementation as well as helped to discriminate between staff roles. Finally, infrastructure was not built to monitor the implementation process, wherefore some matters were dealt with as they occurred.

Summarizing the steps taken during phase 1, one point might be that the organization had faith in the intervention and respect for its efficacy due to previous positive research results. This might have hampered the implementation as some hurdles might have been over-looked. Overall, the organization appears to have been in an early phase regarding readiness to change, although interest, buy-in, dedication, skills, and guidance were satisfactory and favorable.

Phase 2: creating a structure for implementation

In notes from start-up meeting, team meetings and unit meetings it appears as if team members invested time and effort into planning the intervention content and design. They also followed up on their responsibilities and were ambitious in performing tasks before deadlines and detecting problems as they occurred. However, missing actions in phase 2 refers to some of the over-looked actions in phase 1. Foreseen challenges were not addressed proactively. Implicit responsibilities and limited previous experience from hybrid trials sometimes hampered teamwork.

Phase 3: ongoing structure once implementation begins

The cost evaluation plan shows that a considerable part of funding was aimed at technical support and additional funding was added to this post throughout the implementation process. Also, senior supervisors with expertise were connected to the team. Expertise in implementation theory and skills relating to collecting process data, evaluating implementation data and present such feedback, was however not added to the team until phase 4. Also, the need for scheduled support to deal with challenges and resolve conflicts did not become apparent to the team members in time. An infrastructure for communication was in place (team meetings). However, problems still occurred due to culture barriers between technical staff and design staff. Besides difficulties to bridge occupational-specific languages, over-confidence in one’s ability to understand the other party’s needs and prerequisites might have had an effect.

Phase 4: improving future applications

Extensive actions were taken in phase 4, such as sharing experiences with various stakeholders, in written reports and in oral presentations at conferences. Knowledge has been transferred to similar projects and some collaborative relationships exist. The many steps taken in this latter phase might be explained by the organization’s vision to implement digital care and the perceived needs both by consumers and front-line staff. Also, the host setting is a clinical department at a university hospital with the responsibility and ambition to produce and disseminate research. As implementation research may still be regarded novel in this field, conceptualizing experiences may be essential to ease thorough assessments in phase 1 of future initiatives.

Key process challenges and steps that might need further attention considering condition-specific barriers and IPRP as host setting, are presented in Figure 3 (i.e., Logic model). These could be applied in the next step of dissemination, for example if IACT is given before or after IPRP, preferably in a type 2 hybrid effectiveness-implementation study, where the implementation strategy and clinical intervention are tested simultaneously [48].

Principal findings

This retrospective report aimed to describe what actions affected the implementation process of IACT as an added psychological treatment alongside IPRP. The appropriateness of IACT for chronic pain is worthy of investigation as IACT has not yet been disseminated in chronic pain clinics as it has in psychiatric care. The complex feature of pain rehabilitation programs could add to explain some of the difficulties to align IACT to a clinical setting where multiple professionals, multiple modalities and multiple caretakers are involved. Alignment and other key process challenges found in the results from this study are presented in Figure 3 and will now be discussed.

Infrastructure for meetings and awareness of cultural competences are known as important to facilitate effective communication among team members during implementation, in this case system developers and health care givers. The result in this study shows that frequent team meetings and detailed specifications eased collaboration within the project team. However, both technical staff and clinical staff, were overconfident about their ability to understand the others intention. Together with lack of time for testing as the application was developed, some technical functions caused disturbance even though they were perceived as unproblematic and also did not serve their purpose. Thorough testing, especially of data extraction, is therefore listed as an influential factor in the Logic model. Monitoring adaptions and collecting implementation data during the initiative are also noted as key process challenges.

The steps in the first QIF-phase details how to assess readiness. However, certain aspects of readiness, as incentives for caregivers, fitting with organizational needs, staff competence and end-users’ motivation, might be more difficult to assess compared to an organization’s vision or patients’ benefits from a caregiver point of view. One potential hurdle in the present implementation process might have been that part of the context was not yet fully prepared for change. Perhaps, this might be more common in health care services where tradition, routines and experience are highly valued as it preserve stability and continuity. The theory of diffusion of innovation [58] might help to explain the diversity of readiness within an organization, where the research team members might be innovators and early adopters, while the greater part of caregivers and staff, approximately four out of five, are followers, late adopters or laggards.

Descriptive analyses, appraisals, comparative case studies and guidelines for integrating internet delivered psychological interventions in routine care are now emerging [19], [20], [21], [22]. The present study uses a different approach to improve the planning and design of future IACT for chronic pain, namely an implementation science perspective focusing on contextual factors and caregivers as stakeholders. The present project stems from av line of earlier [12, 16, 38] and similar projects [35, 36] and builds on to a growing body of evidence. The unique contribution and novelty of the present study is that the result is presented in a logic model for future implementation projects. Logic models are a foundation in implementation work and there is to date no similar description in the literature to take guidance from when planning and designing implementation of IACT for chronic pain. The detailed descriptions of possible influential factors and potential key process challenges makes a logic model valuable and relevant for clinicians in the progress of designing and implementing IACT for chronic pain.

Strengths and limitations

The QIF was chosen as framework for this retrospective evaluation of a hybrid 1 effectiveness-implementation study because of its suitability when evaluating the implementation of a novel intervention into an established setting; IPRP. Noteworthy is that IPRP is a complex intervention, intertwined with other on-going processes as pharmaceutical follow ups and the sickness benefits system. The QIF provides a comprehensive set of questions that focus on actions, which helps in both planning and evaluation of an implementation process. However, there are some methodological concerns in need of attention.

The present QIF-analysis was led by a project team member acquainted with the cultural preferences of chronic pain patients and with knowledge of the intervention and the setting, to ensure validity in innovation fit [49]. The categorization was done in collaboration with a researcher with comprehensive expertise in internet delivered interventions, which allowed for comparisons with ICBT and IACT implementation in other fields, besides recognizing blind spots. However, the contribution from a second co-rater, independent from the initiative, with expertise in implementation research could potentially have yielded different and complementing information and enhanced the study’s credibility through analyst triangulation.

No quantification of the collected data was done, as the main purpose was to find implementations actions, evaluate their impact and create a logic model. However, quantifications are important for comparisons with similar initiatives, measuring success and evaluating changes made to improve dissemination. Although, when measuring adoption and appropriateness, qualitative data might contribute more.

In spite of these limitations, there are also some strengths. The QIF enables triangulation by gathering of data from several sources, in this case different kinds of documents, which speaks for the study’s credibility. The specific questions in the 14 steps in the QIF analysis, guides data coding and enables other researchers to repeat the analysis. Also, the specificity of the questions helps the rater to stay consistent with the raw data, which speaks for the study’s dependability, as do the detailed flowchart of analysis (Figure 1: Flowchart of analysis process) together with Table 1 and Figure 2, which allowed for an inquiry audit by a researcher with expertise in IPRPs. Concerning confirmability, an audit trail was not applied in this study, as the QIF already provides a rationale for coding. However, Figure 1 shows the rationale for decisions made during the process and constitutes a mean for reflexivity.

Another strength is that this study evaluated an implementation process in a regular clinical setting for patients with chronic pain and comorbidities. Also, the intervention was given by host staff. Stakeholders were involved and all the usual matters of clinical service were present, as sudden shortage of resources, prioritization changes and staff transfer. For example, infrastructure for communication had to be aligned with other clinical processes. Also, recruitment of clinical key champions, who promotes the initiative at host-level, may have been affected by organizational structure and culture. This makes this implementation study representative of some aspects of the real-world experience of caregivers in clinical practice, implementing IACT into IPRP.

However, the benefits of the clinical setting may compromise the transferability of the findings. To make the logic model as clinically useful as possible it contains thick descriptions of the initiative, specifying influential factors, resources and desired output, outcomes, and impact. Hence, the applicability is limited compared to a generic model transferable to chronic pain rehabilitation settings in general. However, the purpose was to specify key processes for implementation of IACT in IPRP, which is widespread and well-established in Sweden [59] and other Western countries.

Recommendations for future research

There are examples of process evaluations of internet delivered interventions where a TMF has been applied when IACT was added to routine care [60]. However, more usual is that evaluations are done retrospectively using qualitative analysis [61]. When moving on to hybrid-2 studies where implementation is studied along-side effectiveness trials in clinical settings, there are benefits of applying a TMF from start. A TMF provides means to plan, monitor, collect data, evaluate the process, and adopt the initiative as the implementation occurs.

One observation concerning QIF though, is that it does not specifically invite the researcher to use lessons learned from previous initiatives. Considering the ongoing digitalization of rehabilitation for chronic pain, one way to ease implementation may be to focus on population-specific barriers, especially negative effects of treatment and risks for attrition [38, 62]. Such known hurdles may point to which implementation actions need extra attention. Also, a research project controller may continually ensure that assigned responsibilities with aligned authority are sufficient to foresee and detect hurdles and adopt the initiative in time.

Considering the many factors that may influence implementation; host setting, caregivers, end-users, stakeholders, intervention and format, logic models may be helpful to target key process challenges, meanwhile considering resources, assumptions and relevant outputs, outcomes and impacts for the specific intervention, format, diagnosis and context. Potentially, sustainability in implementation depends on an ongoing change process that ensures the continuity of facilitating implementation actions that maintain factors crucial to success, although flexible to the changing needs among organization, provider and end-user. The logic model presented here (Figure 3) attempts to address some of those key process challenges.

Relevance to clinical practice

To move implementation science forward, attention to implementation outcomes enlighten us on the effect of implementation actions. In the present study we focused on adoption of IACT for chronic pain, its appropriateness and actions relating to sustainability. We suggest that sensitivity to the changing needs in a clinical setting is evident for sustainability of IACT for chronic pain, along with the use of a TMF in the purpose to plan and evaluate not only the intervention but also the implementation process. The result show that sustainability may be challenged by unfitting capacity building in QIF phase 1 and insufficient attention to structure in phase 2. Appropriateness to the organization was facilitated if infrastructure for communication (in phase 3) resembled the already existing structure in the organization and challenged by an insufficient needs assessment in phase 1. Adoption was facilitated by involving staff key champion in all QIF phases and the ability to adapt to occurring hurdles during phase 3.

The primary implementation outcome in this study was appropriateness for organization. When summarizing the results, two ideas to better the appropriateness of IACT for chronic pain emerge, concerning aim and timing of IACT for chronic pain. First, psychological interventions in IPRPs are primarily group based, with potential to affect pain-induced anxiety and depression [63], although psychiatric conditions are not specifically targeted. ICBT has been found helpful in treating comorbidities alongside face-to-face treatment [64]. If thoroughly aligned, the transdiagnostic approach of IACT may also be helpful as an add-on individual psychological treatment focusing on psychiatric comorbidity alongside IPRP.

Second, the next step of implementation in clinical pain services may be to add IACT before or after IRPR. As a pre-treatment, IACT could potentially prepare patients for face-to-face sessions, help with setting goals, teach key concepts and basic ACT skills, give psychoeducation and explore expectancies for IPRP [22]. A second alternative may be to add IACT after IPRP, to encourage continuous self-care by applying learned skills [16] and by sharing treatment content with significant others to create a supporting environment [45].

Conclusions

The present study adds to a growing body of knowledge as it suggests key process challenges, hypotheses, and potential influential factors in a Logic model (Figure 3), from an implementation science perspective. The first conclusion from the present appraisal is that a TMF may be beneficial to align IACT and IPRP for a successful implementation. A second conclusion is that there might be condition-specific key implementation actions, that either facilitate or challenge implementation. Lastly, previous successful implementations from related fields suggest that IACT may work as a complementary tool to treat comorbidities individually, wherefore we suggest that a sequential approach where IACT is given before or after IPRP may be a possible next step.