Electronic ordering and the management of treatment interdependencies: a qualitative study of paediatric chemotherapy

Background There are serious safety risks associated with chemotherapy, often associated with interdependencies in regimens administered over months or years. Various strategies are used to manage these risks. Computerized provider order entry (CPOE) systems are also implemented to improve medication safety. Little is known regarding the effect of CPOE on how clinicians manage chemotherapy interdependencies and their associated safety strategies. Methods We conducted a multi-method qualitative study in a paediatric hospital. We analysed 827 oncology incidents reported following CPOE implementation and carried out semi-structured interviews with doctors (n = 10), nurses (n = 6), a pharmacist, and oncology CPOE team members (n = 2). Results were interpreted according to safety models (ultra-safe, high-reliability organisations [HROs], or ultra-adaptive). Results Incident reports highlighted two interrelated types of interdependencies: those within organisation of clinical activities and those inherent in chemotherapy regimens. Clinicians reported strategies to address chemotherapy risks and interdependencies. These included rigid rules and ‘no go’ contexts for treatment to proceed, typical of the ultra-safe model; use of time (e.g. planning only so far ahead) and sensitivity to operations, typical of HROs. We identified three different time horizons in CPOE use in relation to patients’ treatments: life-long, the whole regimen, and the ‘here and now’. CPOE supported ultra-safe strategies through automation and access to rules/standardisation, but also created difficulties and contributed to incidents. It supported the ‘here and now’ better than a life-long or whole regimen view of a patient treatment. Sensitivity to operations was essential to anticipate and resolve uncertainties, hazards, CPOE limitations, and mismatches between CPOE processes and workflow in practice. Conclusions Within oncology, CPOE appears to move the ‘mix’ of risk strategies towards ultra-safe models of safety and protocol-mandated care. However, in order to operate ultra-safe strategies embedded in CPOE and stay on protocol it is essential for clinicians to be thoughtful and show sensitivity to operations in CPOE use. CPOE design can be advanced by better consideration of mechanisms to support interdependencies.

in use in a paediatric clinical oncology unit, with a focus on the management of interdependencies. Our objectives were to identify the strategies clinicians apply to safely manage the interdependencies inherent in paediatric chemotherapy, and whether/how CPOE affects these.
For this study, we drew on theoretical and practical insights from Vincent and Amalberti's research on safety strategies 'in the real world', and the three main safety models they propose -ultra-safe, high-reliability organisations (HROs) and ultra-adaptive [3,25,26]. We briefly explain their framework in the next section, as we describe our methods and provide definitions.

Analytical framework
Vincent and Amalberti [3] explain the different 'real world' safety strategies applied in different sectors or organisations for dealing with risks. Such strategies may involve, for example: 1. eliminating exposure to risk by defining 'no go' contexts for operations (following Amalberti, we refer to this as 'Plan A' [25] in aviation, equivalent to grounding all flights when a volcano erupts) and applying rigid standard operating procedures; 2. introducing barriers to risks by engineering optimisation of work processes (including by introducing technology); 3. dealing with risks by improving organisational capacity for monitoring, adaptation and response (e.g. by maintaining 'sensitivity to operations' [27], or awareness of how one's actions affect others); and 4. developing professional expertise for extreme situations. We considered these strategies as different ways, at different levels, to manage risks associated with interdependencies. These strategies also approximately correspond to three models of safety in organisations ( Fig. 1)-the ultra-safe (examples 1 and 2 above), high reliability organisation (HRO) (example 3) and ultraadaptive (example 4) [3,25]. Different healthcare contexts have a different combination of the three models, with some working more often than others as ultra-safe, HRO, and/or ultra-adaptive, along a continuum. Thus, it may be expected that: 'A healthcare team might, in one afternoon, work in an ultra-safe manner at some points, such as when a care pathway is clearly defined and entirely appropriate for the patient; they may work in a high-reliability mode for the main part and, for short periods, in an ultra-adaptive mode.' [3] Building on Thompson's Organization in Action [28], we define interdependency as the relation between tasks or activities, where related tasks/activities must all be completed in a timely fashion, reliably, and/or safely, to achieve safe processes and/or outcomes. While many interdependencies are sequential (one task contributing to the next; a task cannot be completed unless other tasks have been completed first), others may be reciprocal (one task contributing to the other and vice-versa), or pooled (each activity independently contributing to the whole) [28]. For example, nurses' chemotherapy administration depends on doctors' prescribing, which may depend on pathologists' reporting results (sequential tasks); supplying medications in hospital pharmacies on a just-in-case basis, contributes to doctors prescribing medications to patients, while doctors prescribing also calls for medications being stored in pharmacy on a just-in-case basis (reciprocal relationship between activities); repeated administration of chemotherapy doses to a patient cumulates towards a safe maximum level (pooled activities). We define time dependencies as those where safety in management of the interdependency requires attention to timeliness or timing of related tasks (e.g. administration of sequential doses at specific time intervals).  [3,25], with additional reference to collective mindfulnesstypical of high reliability organisations models of safety [27]. Collective mindfulness manifests when people on the frontline collectively show preoccupation with failure (ongoing wariness that errors are possible), reluctance to simplify interpretations of unexpected events (questioning assumptions, uncovering blind spots), sensitivity to operations (having an integrated understanding of current situation, e.g. awareness of how one's actions affect others), commitment to resilience (awareness that it is impossible to anticipate all situations, needs for adaptation) and deference to expertise (persons with expertise to make decisions regardless of hierarchy) [27] Setting and design The study took place in the oncology unit of a 350-bed tertiary paediatric hospital in New South Wales (Australia) between September 2018 and June 2019. The oncology unit is the largest children's cancer unit in the State, with up to 150 new referrals each year. The unit has an inpatient ward plus an outpatient clinic run as an 'open door' service where patients and their families can be seen without an appointment. Doctors are routinely called to see patients not under their regular care.
Eighteen months prior to this study, the hospital had implemented hospital-wide CPOE (Cerner [29]) for prescribing and administration of medications, test orders and results, as part of an electronic medical record system. The oncology unit made use of CPOE functionality for linking orders ('PowerPlan') to incorporate chemotherapy regimens into the system and to prescribe and administer chemotherapy for individual patients. A chemotherapy PowerPlan team was responsible for building chemotherapy protocol templates in the CPOE and training oncology clinicians in use of the system.
We carried out a qualitative, interpretative study combining incident report analysis and interviews. Analysis of the incident reports informed the interview questions, and the insights gained through the interviews fed back to further analysis of the events described in the incident reports.

Data collection
Incident reports were submitted by staff through the hospital online incident reporting system. Any event that resulted in (or had potential to lead to) injury, damage or other loss is required to be reported. The hospital provided us with all incident reports concerning oncology patients recorded in the period 15 August 2016 to 15 February 2018 (the first 18 months following CPOE implementation in oncology), whether or not these were specifically related to medications or CPOE.
Interviews were carried out by a researcher (VL) with a background in health informatics and qualitative research. All clinicians in the oncology unit and members of the chemotherapy PowerPlan team were invited to participate as we were seeking to maximise variety of levels of seniority and CPOE expertise. Access to participants was facilitated by the unit coordinator. We sought participants for interviews until we reached code and meaning saturation [30]. Interviews were semistructured, using questions (Additional file 1) aimed to uncover how clinicians deal with interdependencies in the medication process, and whether the CPOE supported their work. With participants' written consent, interviews were audio recorded and professionally transcribed.

Data analysis
We carried out a qualitative content analysis [31] of the text narratives of the incident reports (fields for Incident description and Contributing factors) and the interview transcripts. Analysis was carried out with support of NVivo v.11, by one researcher in discussion with coinvestigators. The free text fields from incident reports were treated as narrative accounts of events from frontline clinicians [32,33]. All reports were included, but only those related to direct patient care coded. We took both a conventional (inductive) content analysis approach, with open codes derived from the data, and a directed (deductive) approach with our objectives providing higher level categories for grouping the open codes.
Analysis of interview transcripts was carried out iteratively. First, an in-depth detailed understanding of the data was gained through careful reading and line by line coding. Open codes were applied through both conventional and directed approaches, to address study objectives and capture significant aspects of the context. Chemotherapy medication processes, as described by clinicians in the unit, were represented in flowcharts to better understand interdependencies in workflow. This analysis was done concurrently with, and soon after, data collection. A framework analysis approach [34] was then applied using categories from Vincent and Amalberti's model.
Throughout the analysis process, discrepant findings were sought. Findings from incident analysis were triangulated with those from interviews to enrich understanding [35]. The head of the oncology unit (LDP) checked 'the "fit" between respondents' views and the researcher's representation of them' [36] (credibility).

Results
A dataset of 827 incidents were received from the hospital and included in the analysis. Incidents not concerning direct patient care were then excluded (n = 73). (Details of the 827 incidents are given in Additional file 2 and elsewhere [37]). In the analysis of the incidents, the CPOE appeared to be both a mechanism for safety and a contributing factor for incidents.
Twenty interviews were conducted with 19 participants (Table 1). Interviews lasted about 30 min.
We integrated results from interviews and incidents analysis and structured these around interdependencies and risks to medication safety, strategies to deal with these risks, and the role of the CPOE for each strategy. We identified quotes from interview transcripts by interview number (idX), but do not provide details on interviewees' roles to protect their anonymity. We refer to incident reports by their row in the dataset (iX).

Interdependencies and risks to safety with chemotherapy
Work practices related to paediatric chemotherapy treatments were rich in interdependencies. Across interviews and incidents, we identified two interrelated overarching types of interdependencies that characterised chemotherapy prescribing and administration: first, those related to the organisation of clinical activities, in particular the medication workflow involving doctors, nurses and pharmacy staff, and its interplay with a variety of hospital services (e.g. intensive care, pathology, imaging, as well as other hospitals), and second, interdependencies inherent to chemotherapy regimens that dictated combinations of medications and tests. Time dependencies were especially apparent, such as precise time gaps between administration of successive doses.
Both types of interdependencies (organisational and regimen-related) were identified in incident reports and often involved CPOE. For example, an incorrect date in a CPOE prescription and a delay in documentation of the patient being 'ready for chemotherapy' (organisational dependency-medication workflow) delayed administration due to the medication requiring pre-medication (a regimen dependency), which further cascaded into requiring monitoring of the patient 'after hours', leading to increased risks for the patient and costs to the hospital (organisational dependency).  Ultra-safe risk strategies applied to chemotherapy processes Chemotherapy safety risks were reported to be managed through a range of strategies based on the application of rules and multiple safety checks. These appeared more typical of the ultra-safe model of safety [3] than the HRO model.
Specifically, participants spoke of rigid rules regarding who was authorised to prescribe chemotherapy (on CPOE) and the content of medication orders as per chemotherapy protocols. Continuing with a regimen (e.g. progressing to the next cycle) was organised around the principle of withholding treatment until the patient recovers (comparable to 'Plan A' in the ultra-safe model).
Participants reported multiple checks during the medication process. In particular, nurses were tasked with checking each dose against the protocol to alert doctors to any discrepancy. This was 'an institution specific practice' (id16) that was maintained after CPOE implementation, despite the use of pre-set CPOE templates that would (or should) support doctors to produce orders matching the corresponding protocol.
...when I am ordering it [on CPOE], it looks fine to me.
[But] The nurses [would] say, "Well, can't do that because it is out of sequence," or, "you have pushed this onto that." [...] Or they will say, "The protocol mandates 120% of this and you are giving 110% or 150%." And they are just simply saying, "Is this what you want me to do?" ... (id16) Sometimes the patient's condition led to clinicians deciding to deviate from the protocolsuch as when 'this child has such a high-risk disease that no matter what I'm going to go ahead' (id16) despite test results not reaching the threshold indicated by the protocol. This kind of judgementhighly dependent on individual clinical expertise -typically belongs to the ultra-adaptive model of safety. However, in this contextwhere the doctor had to communicate the decision to the nurses and to other doctors in order to act on itit is perhaps suggestive of a HRO approach to dealing with risks.
Does CPOE support an ultra-safe model of safety with chemotherapy?
CPOE supported strategies typical of the ultra-safe model through hard and soft mechanisms, namely through automation, access to information and standardisation of the semantics of protocols (disambiguation) ( Table 2). In terms of automation, the CPOE provided clinicians with all required medications pre-built in templates, to reduce the possibility of medications being omitted by mistake, which were also linked with appropriate time dependencies (if doctors delayed one, the others would be automatically delayed for a corresponding period). The CPOE provided automatic dose calculations and warnings based on embedded rules, and 'exposed' unreliable patient weight and height data used for dose calculation, by displaying trends and normal ranges. CPOE automation also limited prescribing and administration rights through different users' profiles; the embedded workflow management system automatically routed the orders to the authorised person. The CPOE paused the workflow proceeding forward to administration until the system had been notified via a checkbox that the patient was 'ready for chemo'.
However, these CPOE contributions towards an ultrasafe approach to chemotherapy were challenged by the perceived complexity of operating the system (especially when protocol templates were not available in the system to prescribe a patient's regimen), and difficulties in identifying 'where the patient is at' in the protocol. In terms of 'soft' mechanisms, we identified two aspects of CPOE that supported use of ultra-safe risk strategies. All clinicians had access both to patient information and the relevant chemotherapy protocol; previously, paper-based protocols were kept in drawers and not easily accessible across place and time. In addition, the process of converting paper-based protocols into electronic versions brought to the surface previously invisible ambiguities in their rules. To automate these rules, these had to be clarified and uniquely defined so that all clinicians would interpret the protocol in the same way.

HRO strategies -management of uncertainties and 'usual hazards' through adaptation and different time horizons
The application of the principle of withholding treatment until the patient recovers ('plan A') meant that planned doses or cycles often needed to be rescheduled. Thus, for any patient there would be uncertainty about when exactly the treatment would be given. This schedulingor rescheduling -of the treatment must also take into account the constraints posed by the hospital's resources (organisational interdependencies), for example 'our general anaesthetic days are Monday and Thursday' (id16), or over holidays, 'services aren't working quite as well' (id1). The patient and family may also pose constraints on suitable datessuch as wanting to avoid school photo day.
Clinicians reported that it was challenging to plan cycles, tests and doses much in advance. Although the CPOE theoretically allowed clinicians to prescribe a whole regimen in advance, given these uncertainties, interviewees indicated that they would only schedule about a month ahead in practice.
Researcher: the doctors might schedule the entire one year of treatment in advance? Participant: [...] they probably wouldn't. [...] we would chart chemotherapy, kind of, a month out, because [children] can grow so much in such a short time, the doses need to change. Other things can happen as well that they may start to not recover as well, so we need to modify the dose, it was too big for them, so we need to back off [...]. (id7) Doctors therefore made use of time as a risk management strategy [25] to deal with the uncertainties. More specifically, we identified three time horizons used to approach a patient's chemotherapy regimena life-long view, the whole regimen view, and the 'here and now'. CPOE automation better supported the 'here and now' than the longer time views (Table 3). For example, clinicians reported difficulties with finding and collating information to calculate a patient cumulative dose (a lifelong view), and with the fragmented electronic display of the whole regimen.

Sensitivity to operations and CPOE
We found that a combination of situation awareness, organisational awareness and CPOE awareness was perceived as essential for safe and efficient CPOE use (Fig. 2). Clinicians had to maintain (and act on) situational awareness to understand 'where is the patient at' in their specific regimen and with respect to the corresponding protocol. Several interviewees referred to this information need -'where patients are up to' (id1, id5, id7, id8-12). Clinicians also had to maintain (and act on) organisational awareness. For example, they had to act on the system with awareness of self and others 'down the line' (Table 4). They had to be aware of CPOE, learning to be watchful of its automated behaviour such as automated recording of times and dates. With use, clinicians learned that, for a variety of reasons, time and dates in the system may not accurately reflect the times when medication activities took place. This had repercussions for regimen time dependencies, which were also encoded in the templates.

Discussion
We studied a paediatric oncology unit using a hospitalwide CPOE system, and identified two inter-related overarching types of interdependencies associated with chemotherapy: those related to the organisation of clinical activities, and those inherent to chemotherapy regimens in terms of dictating precise combinations of medications, and combinations of medications and tests. Time dependencies were especially apparent across both interdependency types. The time dependencies in a patient regimen were dealt with through a cognitive strategy known as 'fragmentation' ('allowing shorter horizon planning' [39]) and a practice of temporary plans and constant adjustments. These we associate with 'adaptation' strategies typical of HRO models of safety.
CPOE automation seemed to better support the small scale, short-time regimen dependencies that are largely under control of the team, rather than the scheduling of services where organisational dependencies are not under their control. It is possible that the small scale, short-time regimen dependencies were mainly sequential interdependencies (e.g. administration of different medications as sequential steps), while the organisational dependencies were also relational or pooled, and that sequential interdependencies are easier to automate in workflow management systems.
Both ultra-safe and HRO strategies were used to address chemotherapy risks and interdependencies. CPOE automation supported application of ultra-safe strategies in chemotherapy such as compliance with protocols' rules and 'no-go' contexts for proceeding with chemotherapy. CPOE support mechanisms for an ultra-safe model of care were both 'hard' (automation) and 'soft' (information availability/disambiguation). However, prescribing regimens with CPOE was difficult when protocol templates were not readily available; it was difficult to track medication data across screens, and a summary display of the whole regimen (showing protocol variations, if any) was not available; automated time-stamps of medication administration were not always accurate, with repercussions for subsequent time-dependent doses. Therefore, in addition to situational awareness at the level of the patient [38] and awareness of the organisation's teams and services [27], clinicians had to maintain awareness of the technology ('CPOE awareness'), its limitations and consequences. We propose a definition of CPOE awareness as knowledge of how the technology works, how to make it work, and the effects it produces, and acting on this knowledge. Compared to paper-based records, CPOE makes patient information available, independent of time and place (*) Difficulty in tracing specific data across different orders/screens (*) IT teams can manually build into the system automated alerts on cumulative dosesbut this must be done for each drug, and may not be available when needed ...the cardiologist team always need to know how much anthracyclines they've received, because it's cardiotoxic and the dose that they've received makes a difference about what was risk stratifying and things. So to try and find a patient's cumulative anthracycline dose sometimes has been exhausting. More exhausting than night shifts exhausting. (id20) whole treatment A view over 'where we are at' in the protocol as a whole; how the protocol is being givenpossibly adaptedto the patient, and the patient response to treatment. It is a view of how much chemotherapy overall is needed, to minimise toxicity and maximise effect, and how it is given over time a Over a life-long view, there are concerns for life-long risks of cumulative doses. A whole treatment view is required for most treatment decisions; each dose is assessed in view of patient's response to previous doses, in relation to future doses, future options for treatment -at any point in time, the sensemaking is retrospective and prospective. The view over the here and now is concerned with 'the minutiae' of the medication process related to the specific dose, around the time of administration. It's where the order is finalised, multiple checks performed, medications given (or not)a time that may span over a few days The finding that 'CPOE awareness' was needed in this setting is consistent with other CPOE chemotherapy implementations that required significant investment of time and resources to enable clinicians to gain necessary familiarity with all the nuances of CPOE workflows [19]. Our finding that CPOE automation supported application of ultra-safe strategies in chemotherapy is in line with existing literature that shows how CPOE may improve compliance with chemotherapy protocols [17]. However, prescribed regimens are subject to change and in need of adaptation, also due to application of ultrasafe rules (typically, waiting for patient recovery before proceeding with treatment). In the late 1990s, medical sociologists Timmermans and Berg explained how standardisation of patient care with (paper-based) oncology protocols was achieved through clinicians' 'active (not mindless) support [...] to maintain the protocol's trajectory on course' [40]. In their account, adaptation was an enabler, rather than opposer, to standardisation. We posit that active (not mindless) adaptation requires situational awareness (knowledge of 'where is the patient at' in relation to a prescribed regimen and underlying protocol), which must be facilitated by CPOE displaying interdependent elements of regimens over different time horizons (in chemotherapy, known as 'roadmaps' [12]). This is also supported by recent research suggesting that chemotherapy errors with CPOE may be prevented with knowledge of a patient's chemotherapy history and in-depth knowledge of protocols [41]. Nurses' and/or pharmacists' routine checks of CPOE orders against protocols [22,24] also reported in the unit we studied -are one way to address CPOE limitations by automatically monitoring variations to protocols, and may also be facilitated by the availability of electronic roadmap displays.
Strengths of this study include multiple sources of data -interviews and incident reports -and analysis informed by theory. A limitation is that we did not observe the activities described. Instead, we relied on participants as 'their own ethnographers' [42]. We had limited nurse participation. We identified ultra-safe and HRO strategies; we did not find any strategy from the ultraadaptive safety model possibly because we did not observe activities. Patient safety incident reports were brief and provided only limited information; incidents related to multiple aspects of patient care, not exclusively chemotherapy. We took a qualitative approach to incident analysis given the limitations of analysing these quantitatively; reported incidents are likely to represent only a fraction of the incidents that occur and thus numbers of incidents cannot be relied on to establish frequency [43]. The CPOE system investigated was a general system adapted for chemotherapy, rather than a bespoke chemotherapy system; it is not known whether the findings generalise across other organisations and other systems.  [38] as "the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future", applied to a patient. With organisation awareness we refer to the understanding of how roles and services fit within the organisational structure, institutional practices and rules, and how to operate within these practices to achieve desired effects. This definition includes individual awareness of local teams and workflows, or of the wider hospital organisation, within or beyond individual's control. CPOE awareness is knowledge on how the technology works/how to make it work/the effects it produces and acting on this knowledge. We take sensitivity to operations (at individual or collective level) as any and all of the three types of awareness at any one time Ultra-safe services 'are highly standardized and rely heavily on automation and information technology' [3]. Implementation of CPOE in chemotherapy appears to be a move towards ultra-safe, but our findings suggest that CPOE design must be improved. In such a complex and high-risk setting, CPOE design should facilitate clinicians' decision-making processes, rather than add difficulties. Lessons can be learned for design of chemotherapy CPOE that better supports the management of interdependencies in regimens and workflows [21]. This might include affording a variety of visualisation displays over different time horizons and capturing more accurate timestamps of activities, tracking protocol variations and cumulative effects over time. CPOE implementations also need to support learning processes for clinicians to gain the awareness needed to use CPOE systems safely. For this, sufficient enabling resources (e.g. staff/patient ratio, CPOE training and assistance, quiet time for prescribing) must be in place at the point of roll-out and maintained throughout staff turnover [19]. To our knowledge, this is the first study to apply Vincent and Amalberti's models of safety and related risk strategies [3] to evaluate a technology implementation in a healthcare setting. The study has been able to identify mechanisms by which CPOEs can mitigate or exacerbate medication safety risks.
Patients and families may have different priorities or values about medication safety and compliance with rules, that may vary during the long period of chemotherapy treatments. Future research in paediatric oncology should investigate whether/how patients and families' priorities or values change the way clinicians use the CPOE for chemotherapy regimens, and how they approach the different time frames: the 'here and now', 'whole treatment' and 'life-long'. Further research with CPOE for chemotherapy should also investigate time dependencies in regimens in more detailincluding how nurses manage administration times between doses and automated timestamping of activities.
Conclusions CPOE appears to affect the 'mix' of risk strategies in place in an oncology unit. It can drive ultra-safe models of safety and protocol mandated care, but operating ultra-safe strategies embedded in the CPOE and staying on protocol also requires HRO strategies including 'sensitivity to operations' in CPOE use. CPOE implementations need to support the processes of learning required for clinicians to gain such collective awareness, and shortcomings in CPOE design must be addressed for it to fully contribute to the ultra-safe.
Additional file 2. Incident analysis -methods and summary of findings.
Abbreviations CPOE: Computerized provider order entry; HRO: High-reliability organisations