Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Effects of a computerised guideline support tool on child healthcare professionals’ response to suspicions of child abuse and neglect: a community-based intervention trial

BMC Medical Informatics and Decision Making volume 19, Article number: 161 (2019) Cite this article

Abstract

Background

Healthcare professionals’ adherence to guidelines on child protection is not self-evident. This study assessed the effects of a computerised support tool on child healthcare professionals’ adherence to the seven recommended guideline activities, and on time spent seeking information presented in this guideline.

Methods

A community-based intervention trial design was applied, comparing access to a paper-based guideline (control) with access to a paper-based guideline supplemented with a computerised guideline support tool (intervention). A total of 168 child healthcare doctors and nurses working in one large Dutch organisation were allocated to an intervention or control group. Outcomes were professionals’ performance of seven recommended guideline activities and the amount of time spent seeking information presented in the guideline. Professionals’ adherence was measured using two methods: health record analysis and a self-report questionnaire. The questionnaire was also used to collect data on the amount of time spent seeking guideline information.

Results

In total, 152 health records (102 in the intervention group and 50 in the control group) were available for analysis. The tool was registered in 14% of the records in the intervention group. Performance of activities, corrected for intentional non-adherence, was except for one activity, high (range 80–100%); no differences were found between the control and intervention groups. Forty-nine questionnaires (24 in the intervention group and 25 in the control group) were analysed. Sixty-three percent of the questionnaire respondents (15/24) claimed to have used the tool. No differences in guideline adherence were found between the two groups. Respondents in the intervention and control groups spent, on average, 115 and 153 min respectively seeking relevant information presented in the guideline.

Conclusions

The results regarding use of the tool were inconclusive as the outcomes differed per method. In contrast to expectations, performance of guideline activities was high in both groups. The support tool may decrease the amount of time spent on seeking guideline information. However, given the high adherence scores and small number of questionnaire respondents, the outcomes failed to reach statistical significance. Future research should focus on studying the effects of the tool after a longer period of availability.

Peer Review reports

Background

Child Abuse and Neglect (CAN) has been recognised worldwide as a serious public health issue [1] that needs to be prevented. As abusive parents, maltreated children, and bystanders frequently do not seek help [2, 3], national policies have increasingly emphasised the responsibility of healthcare professionals, sometimes statutorily [4,5,6]. However, multiple studies have shown that healthcare professionals under-identify cases of CAN or do not always respond adequately to concerns (e.g. [7,8,9,10,11]). As poor recognition and response may result in continued CAN, potentially leading to severe consequences [1], health professionals should be supported around these issues.

One approach to support professionals and improve handling of CAN concerns is to provide them with evidence-based guidelines. In the last decade, guidelines on responding to suspected CAN have increasingly become available [12,13,14], including in the Netherlands. Since July 2013, Dutch professionals involved with children and families have been legally obliged to follow guidelines if they suspect CAN [15]. In 2010, the Dutch Centre for Child Health issued a clinical guideline on early detection and response to suspected CAN (further referred to as the CAN guideline) [16, 17]. This guideline has been developed specifically for preventive child healthcare (CHC) doctors and nurses. These professionals are tasked with the identification and registration of (potential) risks to the health and wellbeing of children. The developers of the guideline listed seven key activities which they deem critical, based on an analysis of scientific literature and consensus among professionals and experts [17]. The guideline, which was made available in a paper-based format, is 170 pages long and is also available online. A 40-page summary, a two-page document listing the key activities and a flowchart were also made available to CHC professionals. The key activities described in the CAN guideline are shown below, presented in the order in which the guideline recommends CHC professionals perform them:

  • Risk assessment based on protective and risk factors;

  • Discussing suspicions with caregiver(s) and/or child;

  • Consulting an in-house expert on child abuse and neglect;

  • Consulting the regional child protection service: the Advice and Reporting Centre;

  • Requesting information from professionals outside the child healthcare organisation who are also involved with the family;

  • Acting: providing support, referring the family to other organisations for support or reporting suspicions to the Advice and Reporting Centre;

  • Monitoring the support that is provided to the family and taking action again if the support is inadequate.

Despite the potential benefits, including more consistent work procedures based on the best available evidence and improved health outcomes [18], adherence to guidelines is generally poor (e.g. [19,20,21]). Two studies have evaluated CHC professionals’ performance of CAN guideline activities. Fleuren et al. [17] documented a range of performance of five key activities varying between 67 and 82%. Konijnendijk et al. [22] reported that the percentage of professionals that performed a key activity in all suspected cases varied between 19.5 and 42.7% for the seven activities.

One approach to promote routine use of guidelines is to introduce a computerised guideline support tool through which guideline information is disseminated in a more user-friendly manner [21]. Paper-based guidelines have been criticised for quickly becoming outdated, and being a suboptimal presentation format [23]. Communicating guidelines through a computerised system with an interface similar to an electronic health record makes the application of guidelines more personal and acceptable at the moment of care [24]. Health Information Technology (IT) systems, such as electronic health records, and computerised guideline support tools, are increasingly being introduced in healthcare organisations to enhance care efficiency, quality, and safety [25,26,27]. The need for health IT to facilitate use of guidelines has been discussed in literature [21, 23, 28]. Previous research has shown that a support tool can improve healthcare professionals’ adherence to guidelines and improve professional practice [29, 30].

We developed a computerised guideline support tool that was integrated in the electronic health record used in Dutch preventive CHC. The tool presents guideline information in a concise manner that is quickly and easily accessible. Guideline information is offered at the time and place of decision-making, which has been described as best practice with regard to design principles for usable decision support [31]. The tool also presents record data regarding risk factors. Furthermore, the tool instructs professionals to plan appointments or to perform tasks within the timeframe that the guideline recommends. Also, it reminds professionals to perform activities and register these activities in the health record by use of electronic alerts, and provides real-time notifications when activities are not performed in time. Electronic reminders may achieve small to modest improvements in clinical behaviour [32]. Overall, the tool aims to promote guideline adherence and uniform registration, and to minimise the time and effort needed to access guideline information. As such, the tool may increase both the quality of healthcare and professional productivity. The development of the tool, which is described elsewhere [33], followed an iterative process involving CHC professionals in all developmental stages. Addressing professional attitudes towards a computerised support tool has been shown to be important for its successful implementation [24, 34].

This study addressed the following question: What are the effects of having access to the paper-based CAN guideline complemented with a computerised guideline support tool, compared to having access solely to the paper-based version, on CHC professionals’ adherence to the guideline and on the amount of time spent seeking relevant information provided by the guideline? In this study, the adherence scores were corrected for intentional non-adherence. Arts et al. [21] concluded in their recent systematic review that deviations from guidelines are often supported by valid reasons and that these intentional deviations can also result in good quality of care.

Methods

Design

A mixed-methods community-based intervention trial [35] design was applied, comparing access to a paper-based guideline complemented with a computerised guideline support tool (intervention) with access to a paper-based guideline only (control). The study was carried out in one large Dutch CHC organisation (GGD Twente) in the eastern part of the Netherlands.

Setting

In the Netherlands, CHC professionals provide routine preventive services to virtually all children (0–17 years) in well-baby clinics and schools. These services concentrate on the optimal growth and development of a child, to prevent the child from developing severe health problems [36]. Except for short-term parenting support, CHC services do not provide treatment [9]. In the Netherlands, CHC professionals are well placed to detect, monitor, and respond to suspected CAN for two reasons. First, they have frequent contact with families and their children for consultation, usually 15 times between the ages of zero and four and five times after the age of four [37]. Second, both disciplines have been trained extensively to develop skills in recognising and managing health, psychosocial, and parenting problems. Doctors and nurses work together daily and intensively in a CHC team. They use an electronic health record to keep track of the child’s development.

Study population and group allocation

The subjects of this study are CHC professionals working at GGD Twente. In January 2014, GGD Twente employed 58 CHC doctors and 110 CHC nurses, divided over 21 teams. Professionals in the same CHC team provide services to the same children in a specific geographically demarcated region, such as a village or a city district. All professionals were female. During the study period (February 2014 – October 2014), GGD Twente provided services to approximately 125,000 children between 0 and 17 years old [38].

All 168 CHC professionals were allocated to one of two groups. When assigning professionals to a group, we addressed three issues:

  • Professionals who provide care to the same children, and thus document client information in the same health records, should be in the same group;

  • Contamination between professionals in the intervention and the control group should be minimised [39]. The risk of contamination is expected to be lower when allocation to the two groups is performed on a team level. Twenty-three professionals were part of two or more CHC teams. These professionals should preferably be in the same group to prevent contamination;

  • Both groups had to represent urban and rural areas in equal proportions.

After making the best possible division of teams into two groups, taking into consideration the issues mentioned above, the two groups were randomly assigned to the intervention (11 CHC teams) or the control groups (ten CHC teams) by flipping a coin. Unfortunately, we could not prevent four professionals being assigned to both conditions.

Intervention group

The intervention group comprised 90 professionals, including the four professionals in both groups: 32 doctors and 58 nurses. These professionals provide services to approximately 60,000 children [38]. Prior to the start of the study, all professionals in the intervention group were informed about the aim and procedure of the study in a meeting, and instructed in the use of the tool via a manual and a web-link with an online instruction video. They were asked to use the tool from February 1st, 2014 onwards. A helpdesk, comprising the first author and one of the application administrators, was available during the study period to assist professionals who had questions or encountered problems using the tool.

A professional with suspicions of CAN could access the tool in the electronic health record and link it to a child’s record. The tool has the following functionalities:

  • presenting guideline information in a concise manner that is easily and quickly accessible;

  • presenting guideline information at the time and place of decision-making;

  • providing professionals with (up-to-date web links to) a clear overview of information, aids, and instructions on how to register information about a child’s situation correctly (i.e. accurate, complete, and uniform);

  • displaying relevant information tailored to the organisational and regional context, such as contact details of the in-house CAN expert and professionals from other organisations involved with the family;

  • displaying relevant data registered elsewhere in the child’s health record, including present risk factors;

  • providing instructions to plan appointments or perform tasks guideline activities within the timeframe that the guideline recommends;

  • reminding professionals to perform activities and providing real-time notifications when activities are not performed in time, through use of electronic alerts. These prompts are initiated by registrations that the professionals made earlier.

Figure 1 displays a screenshot of the overview page of the tool, which shows the clinical pathway on the left side: the activities that CHC professionals should perform, sequenced in the most common and logical order.

Fig. 1
figure 1

Screenshot of the tool: tab containing overview page

Full size image

Control group

The control group consisted of 78 professionals: 26 doctors and 52 nurses. These professionals provide services to approximately 65,000 children [38]. They did not receive any instructions regarding the tool, but were informed about the study.

Measurements

Two outcomes were measured: professionals’ adherence to seven recommended guideline activities and time spent seeking information presented in the guideline. Two methods were used to measure professionals’ adherence: record analysis and a self-report questionnaire. The survey was also used to collect data on the time spent seeking guideline information.

The adherence scores were corrected for intentional non-adherence [21]. Valid reasons for non-adherence included: ‘no longer had suspicions’, ‘a team member performed this activity’, ‘caregiver(s) or the child did not approve’ (applies to the activity ‘requesting information from other professionals outside the CHC organisation who are also involved with the family’), ‘a professional outside the organisation informed the CHC professional’ (applies to the activity ‘monitoring actual execution of an activity promoted by the guideline), and ‘a professional outside the organisation performed the activity or is responsible for performing the activity as (s)he was the first professional addressing concerns of possible CAN’. The reason ‘a team member performed this activity’ did not apply to record analysis, as these records were analysed on a team level.

Health records

In total, the health records of 186 children met the following inclusion criteria: 1) The record has a registration that indicates suspected CAN; 2) The first registration indicating a suspected CAN case was made between February 1st and October 1st 2014. Two cases were excluded because one CHC professional objected to the use of her (anonymised) record data and 32 cases were excluded as content analysis revealed that the situation described in the record was unrelated to CAN. In total, 152 health records were available for analysis: 50 from the control group and 102 from the intervention group. Based on national data on the prevalence of child abuse and neglect [40], it was expected that 3750 children in the study region experience child abuse and/or neglect. Figure 2 displays a flow diagram of the record selection.

Fig. 2
figure 2

Flow diagram of the selection of health records

Full size image

The cases described in the health records were followed for 8 months, starting from the date of the first registration that indicated suspicions of CAN. Personal data in the records were anonymised at both the child and the professional level by the application manager and the second author (RH). Afterwards, the first author (AK) read each record completely and coded information regarding adherence to the seven guideline activities, and gender, age of the child, and use of the tool.

Adherence to key guideline activities

For each case, yes/no scores were used to indicate whether each guideline activity was adhered to, corrected for intentional non-adherence [21].

Other variables

Gender and age of the child were documented for each record. Furthermore, use of the tool was analysed for each record. Use of the tool was assumed when the tool was registered in the record. Subsequently, information about the way suspicions arose was retrieved from the records: through observation, through a story from the child, a story from the caregiver(s), a story from a fellow CHC professional in the same organisation, the story of a professional outside the CHC organisation who is also involved with the child, or through the story of a non-professional (e.g. grandparent or neighbour). In 55 out of the 152 cases, CHC professionals were the first professional involved with the child who addressed concerns of possible CAN: 17 in the control group and 38 in the intervention group.

Questionnaire

The questionnaire included questions concerning background variables, performance of key guideline activities, and amount of time spent seeking information from the CAN guideline (Additional file 1). For all questions, respondents were asked to keep the last suspected case in mind (between February 1st and October 1st 2014). Respondents in the intervention group who reported having used the tool at least once were asked to keep in mind the last suspected case for which they had used the tool. Because the last suspected case could have started, for example, in September 2014, it is possible that respondents had not performed all guideline activities before October 1st. Therefore, if a survey respondent mentioned that she did not perform a guideline activity because she had not yet got to it before October 1st, this answer was coded as a valid reason for non-adherence.

The questionnaire emphasised anonymity and confidentiality. A pilot-test, with two researchers and a CHC professional, was performed to assess the comprehensibility and practicality of the questionnaire. Subsequently, the questionnaire was digitalised using the online survey program LimeSurvey. All CHC professionals were invited, by email, to participate in the questionnaire in November 2014. In the second week after the initial mailing, a first reminder was sent. A second reminder was sent a week before closing the database in January 2015.

Eighty-eight professionals filled in the questionnaire (52%). Subsequently, 39 respondents (44%) were excluded because they did not have suspicions of CAN between February 1st and October 1st 2014. In total, 49 questionnaires were available for analysis: 25 in the control group and 24 in the intervention group. Figure 3 displays the flow of professionals participating and eligible for the survey study.

Fig. 3
figure 3

Flow diagram of questionnaire participants per group

Full size image

Adherence to key guideline activities

The questionnaire was used to investigate whether, and how often, CHC professionals self-reportedly performed each key activity in the CAN guideline in their last suspected case. If respondents did not perform a key activity, they were asked to report why they had not performed the activity. If a reason was regarded as valid, non-adherence was recoded to ‘adherent’.

Time spent on seeking information provided by the CAN guideline

All respondents were asked to indicate the number of minutes they had spent on seeking relevant information provided by the paper-based guideline and/or the tool. In addition, respondents in the intervention group were asked to estimate how many minutes they had invested in learning to use the tool.

Other variables

Respondents were asked to report background variables including their profession, their number of working hours per week, and their years of experience as a CHC professional. As in the record analysis, data with regard to the way suspicions arose and use of the tool were collected.

Data analysis

All analyses were performed using the intention-to-treat approach [41]. This approach allows for non-adherence to the tool by professionals in the intervention group. All records and survey participants were included in comparative analyses, regardless of whether they actually reported having used the tool.

Health records

Record data were entered manually into SPSS, version 24. The second author (RH) checked the analysis of 20 randomly selected records (11%), and did not identify inconsistencies or mistakes. Descriptive analyses were performed to describe both background information and outcome variables per group. Subsequently, independent sample t-tests were carried out to compare adherence scores between groups. P-values of <.05 (two-tailed) were considered significant.

Questionnaire

Questionnaire data were downloaded from LimeSurvey [42] to SPSS. For each condition, descriptive analyses were performed on background variables, adherence to guideline activities, and time spent seeking information provided by the guideline. A chi-square goodness of fit test was performed to test whether the observed proportions for discipline differ from the hypothesised proportion (35% doctors and 65% nurses). Chi-square tests and independent sample t-tests were carried out to identify differences regarding background and outcome variables between the two conditions (two-tailed). Statistical significance was defined as a p-value less than .05 (two-tailed). The data related to time spent seeking information were not normally distributed. Therefore, a non-parametric test was performed (Mann-Whitney test) to test differences between the control group and the intervention group.

Results

Background characteristics

The children (79 girls and 73 boys) whose records were included were on average 6.5 years old (SD = 5.2, range 0–17). The records in the intervention group (n = 102) on average concerned older children (M = 7.3 years old, SD = 5.1) than the records in the control group (n = 50; M = 5.1 years old, SD = 5.2) (p = .01).

The descriptive statistics for the background variables of questionnaire respondents with suspicions of CAN are presented in Table 1. Most of these respondents were nurses (67%), as expected based on the proportion of nurses and doctors in the organisation (χ2 (1) = 12, p = .73). Furthermore, most respondents provided services in well baby clinics (49%), 33% provided school healthcare, and 18% provided both services. The respondents had on average 19.5 years of work experience (SD = 8.3) and worked on average 21.7 h per week (SD = 5.0). The 49 participants with suspicions of CAN had on average 2.3 suspicions of CAN. No significant differences in background variables were found between the groups.

Table 1 Background variables of questionnaire respondents with suspicions of CAN in the previous 12 months (n = 49)
Full size table

Use of the tool

In the intervention group, the tool was used in 14 out of the 102 records (14%). According to the questionnaires, 15 out of 24 respondents (63%) reported having used the tool in the intervention condition.

Adherence to key guideline activities

Neither the record analysis nor the questionnaire analysis showed significant differences in adherence scores between the respondents in the control group and the intervention group. Table 2 shows adherence scores based on the record analysis and on the questionnaire data. Self-reported adherence rates were on average high. Except for the activity ‘risk assessment’ and the activity ‘in-house consultation of the CAN expert’, these outcomes are consistent across the two methods used. Risk assessment was virtually never registered in the records, while all respondents in the questionnaire reported that they had performed this activity in the suspected case they answered questions about. Adherence to consultation of an in-house expert on CAN was on average less frequently reported in the records (84%) than in the questionnaire (96%).

Table 2 Adherence to key CAN guideline activities (%, n) per method
Full size table

In this study, valid reasons for non-adherence included ‘a professional outside the organisation performed the activity or is responsible for performing the activity as she was the first professional to raise concerns of possible CAN’. Therefore, we also analysed only the 55 records (17 in the control condition and 38 in the intervention condition) of cases in which a CHC professional was the first to raise concerns. In this analysis, the adherence scores were considerably, but not significantly, lower for two activities compared to the analysis of all 152 records: ‘Consultation of the in-house CAN expert’ (47% in the control condition and 76% in the intervention condition; p = .05) and ‘Consultation of the Advice and Reporting Centre’ (65% in the control group and 66% in the intervention group; p = .94).

Time spent seeking information provided by the CAN guideline

Respondents in the intervention group spent significantly less time on the use of paper-based guidelines compared with the control group, respectively 75 min (SD = 54 min) and 135 min (SD = 121 min) (p = .01). The respondents in the intervention group spent an average of 40 min using the tool (SD = 47 min). When the time spent on the use of the paper-based guideline is combined with the time spent on the use of the tool to seek guideline information, the difference between the control group (M = 153 min, SD = 121 min) and the intervention group (M = 115 min, SD = 80 min) is still considerable, but not statistically significant (p = .34). Respondents in the intervention group had to invest time in learning how to use the tool. Respondents in the intervention group spent on average 53 min (SD = 42 min) on learning to use the tool.

Discussion

This study assessed the effects of integrating a computerised guideline support tool into the electronic health record on CHC professionals’ adherence to a guideline on the early detection and response to suspected CAN, and on time spent seeking information presented in this guideline.

Assuming that 3% of the 125,000 children living in the study area experience CAN [40], the number of identified records of suspected cases (152) is much lower than expected. The identification of suspected CAN cases in the electronic health record depended on structured data. Structured data refers to registrations of fixed names for interventions and activities that indicate cases of CAN, such as ‘(suspected) CAN’, ‘worrying situation’, or the registration of the tool. The low number of identified cases may be due to professionals’ preferences around registering suspicions of CAN in the electronic health record using these fixed registrations [43], as CAN is a poorly defined condition, the ‘diagnosis’ is uncertain, and professionals are often ambivalent about using a label that explicitly refers to CAN [44]. It is possible that, as in the United Kingdom [43, 44], Dutch CHC professionals use a wide variety of phrases to indicate suspected CAN, including those that are indirect or euphemistic. As a result, using structured records to identify CAN is likely to result in an underestimation of suspected CAN cases.

The number of identified cases was especially low in the control group. Although the allocation of CHC teams to the two groups was performed carefully, it was remarkable that two-thirds of the CAN records were retrieved from the intervention group and that children in the intervention group were on average 2 years older. More professionals were assigned to the intervention group (90 compared to 78 in the control condition) due to the uneven number of teams that had to be allocated. However, it is not likely that the small difference in the number of professionals in each group explains why twice as many cases were identified in the intervention group. It is also improbable that CAN is more prevalent in the intervention region. Data on registered incidents of domestic violence (including CAN) in the study area in 2014 showed that 51% of these incidents took place in the intervention area [45]. Possibly (training in) using the tool in the intervention group helped to raise both awareness of CAN and attentiveness to possible indicators of it and stimulated professionals in the intervention group to register CAN-related concerns in the electronic health record.

Use of the computerised guideline support tool

The two investigation methods used show inconclusive results regarding use of the tool in the intervention group: 63% (questionnaire) versus 14% (records). There are several possible explanations for these inconsistent percentages. On the one hand, the high usage rate using the survey method may be due to a self-selection bias [46] and social desirability bias [47]. Professionals who used the tool may have been more inclined to participate in the survey and survey respondents may have provided socially desirable answers. These biases probably resulted in an overestimation of the actual use percentage.

In conclusion, actual use of the tool probably lies between 14 and 63%, showing that many professionals did not adopt use of the new tool. Much literature has described the problem of lack of acceptance of technology in the healthcare field and has sought to predict and explain why some individuals adopt while others reject an innovation [48,49,50,51]. Perceived usefulness and perceived ease of use predict a substantial proportion of use or acceptance of health information technology (IT) [48, 51]. Compatibility between health IT and clinical work, individual characteristics, and organisational culture can also influence health IT acceptance [26, 51]. Furthermore, as the time between introduction and evaluation was short in the current study (8 months), innovators and early adopters may have been more likely to use the tool [49]. Use of the tool may be higher if a longer time frame had been chosen between introducing the tool and evaluating its use.

Adherence to key guideline activities

The findings demonstrate that adherence to recommended guideline activities, corrected for intentional non-adherence, was high in both the intervention and control groups. These results were consistent between methods for five out of the seven guideline activities. The activity ‘Risk assessment based on protective and risk factors’ was virtually never identified in the records (intervention and control condition), while all 49 survey respondents mentioned that they performed this activity. These inconsistent results indicate that professionals either do perform risk assessment when they suspect CAN but do not register this activity explicitly in the child’s record or that they provided socially desirable answers in the survey [47]. The adherence score for the activity ‘Consulting an in-house expert on CAN’ was also higher in the survey than in the record analysis. The results do not provide evidence that the support tool improves professionals’ adherence to the CAN guideline.

This study shows that adherence to guideline activities, corrected for intentional non-adherence, was higher than expected based on previous research that did not correct for valid reasons not to perform guideline activities [17, 22]. Non-adherence to guidelines on CAN prevention can be supported by justifiable and thus valid reasons, in concordance with the findings of Arts et al. [21]. It is important to distinguish between reasons for non-adherence that are intentional and non-intentional. Insight into reasons for non-adherence that are not justifiable, such as poor knowledge, fear of consequences, or low confidence in follow-up care [10, 22, 52], provides opportunities for improvement in the quality of care for vulnerable children. Moreover, it makes clearer what activities really need attention from guideline developers.

Time spent on seeking information provided by the CAN guideline

CHC professionals involved in the development of the support tool expected that using the guideline support tool would cost extra time [33]. However, the results regarding time spent seeking information provided by the CAN guideline suggest that the support tool can actually save time. The variation in time between individual professionals was large, which may explain why the difference between the intervention group (115 min) and the control group (153 min) is not significant.

Strengths and limitations

A strength of this study is that the questionnaire sample was representative of the types of employee in the organisation. The response rate for the questionnaire was satisfactory, at 50%. Another strength of this study is that the methods allowed adherence scores to be corrected for intentional non-adherence [21]. Furthermore, the benefit of using two methods to study use of the tool and guideline adherence is that the methods compensate for each other’s biases and allow cross-validation of research findings [53].

The study is limited in four respects. First, both methods contain biases which cannot be completely excluded. The reliability of record data may be limited by poor registration performances [44]. As a result, guideline adherence may be higher than the results of the record analysis demonstrated [44]. In addition to the tendency of survey respondents to provide socially desirable answers, the reliability of self-report survey data is biased by the possibility that professionals who used the tool or find the topic interesting and important were more likely to participate. However, most adherence scores were consistent across methods, indicating reliable results.

Second, using two methods exposed contrary results regarding use of the tool and adherence to one activity, ‘Risk assessment based on protective and risk factors’, making it difficult to draw conclusions from those data.

Third, the performances of professionals in 97 out of 152 records were regarded as adherent because the record revealed that one or more professionals outside the CHC organisations performed, or were responsible for performing, guideline activities. Correcting for this in 97 records has probably led to an overestimation of adherence scores, as analysis of the other 55 cases revealed lower adherence scores for two guideline activities.

Conclusions

As use of the support tool during the study period was low and guideline adherence rates, corrected for intentional non-adherence, were high in both study groups, no conclusions could be drawn about the effects of the tool on guideline adherence. The findings do however suggest that the tool may have the potential to save preventive child healthcare professionals’ time to seek guideline information. As this study was performed in the primary healthcare setting, the results cannot automatically be generalized to other healthcare fields.

Future research should focus on why CHC professionals accepted or did not accept the tool, using a well-regarded theory of technology acceptance such as the Technology Acceptance Model [51]. Research on socio-technical factors is considered vital to maximise the likelihood of successful implementation of digital innovations to enhance the quality of health care [25]. Also, we recommend use of an evaluation approach in which guideline deviations can easily be analysed. For example, Arts et al. [21] suggest the Standardised Clinical Assessment And Management Plan (SCAMP) for this purpose [21, 54]. We furthermore propose further development and evaluation of the tool after a longer period of availability.

Given the low number of identified cases of CAN in the record analysis, many cases of CAN are not detected by child healthcare professionals. It is therefore paramount that child healthcare organisations structurally invest in educating their professionals in recognizing CAN, to keep children safe from abuse and neglect.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CAN:

Child Abuse and Neglect

CHC:

Child healthcare

IT:

Information Technology

SCAMP:

Standardised Clinical Assessment And Management Plan

References

  1. Committee on Child Maltreatment Research, Policy, and Practice for the Next Decade. 4. Consequences of child abuse and neglect. In: New directions in child abuse and neglect research. Washington (DC): National Academies Press (US); 2014. p. 111–74.

    Google Scholar 

  2. Cohen B. Measuring the willingness to seek help. J Soc Serv Res. 1999;26(1):67–82.

    Article  Google Scholar 

  3. Deater-Deckard K. Parenting stress and child adjustment: some old hypotheses and new questions. Clin Psychol Sci Pract. 1998;5(3):314–32.

    Article  Google Scholar 

  4. Berg-le Clercq, T. Prevent and combat child abuse: what works? 2012. https://www.dji.de/fileadmin/user_upload/izkk/Daphne_international_report.pdf. Accessed 30 July 2015.

    Google Scholar 

  5. Cukovic-Bagic I, Welbury RR, Flander GB, Hatibovic-Kofman S, Nuzzolese E. Child protection: legal and ethical obligation regarding the report of child abuse in four different countries. J Forensic Odontostomatol. 2014;31(1):15–21.

    Google Scholar 

  6. Mathews B, Kenny MC. Mandatory reporting legislation in the United States, Canada, and Australia: a cross-jurisdictional review of key features, differences, and issues. Child Maltreat. 2008;13(1):50–63.

    Article  PubMed  Google Scholar 

  7. Fingarson AK, Flaherty EG, Sege SD. Improving physician identification and reporting of child abuse. J Clin Outcomes Manag. 2011;18(4):185–90.

    Google Scholar 

  8. Talsma M, Boström KB, Östberg AL. Facing suspected child abuse - what keeps Swedish general practitioners from reporting to child protective services? Scand J Prim Health Care. 2015;33(1):21–6.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Reijneveld SA, de Meer G, Wiefferink CH, Crone MR. Detection of child abuse by Dutch preventive child-healthcare doctors and nurses: has it changed? Child Abuse Negl. 2008;32(9):831–7.

    Article  PubMed  Google Scholar 

  10. Visscher SJA, van Stel HF. Variation in prevention of child maltreatment by Dutch child healthcare professionals. Child Abuse Negl. 2017;70:264–73.

    Article  PubMed  Google Scholar 

  11. Albaek AU, Kinn LG, Milde AM. Walking children through a minefield: how professionals experience exploring adverse childhood experiences. Qual Health Res. 2018;28(2):231–44.

    Article  PubMed  Google Scholar 

  12. American Medical Association. Diagnostic and treatment guidelines on child physical abuse and neglect. Arch Fam Med. 1992;1(2):187–97.

    Article  Google Scholar 

  13. Prevent Child Abuse Utah. Child abuse and neglect protocol manual. 2006.

    Google Scholar 

  14. Saperia J, Lakhanpaul M, Kemp A, Glaser D. When to suspect child maltreatment: summary of NICE guidance. BMJ (Clinical research ed.). 2009;339:b2689.

    Article  Google Scholar 

  15. Dutch Ministry of Health Welfare and Sport. Basic model for reporting code domestic violence and child abuse (in Dutch). 2012. https://www.rijksoverheid.nl/documenten/rapporten/2017/01/09/basismodel-meldcode-huiselijk-geweld-en-kindermishandeling. Accessed 23 July 2013.

    Google Scholar 

  16. Wagenaar-Fischer MM, Heerdink-Obenhuijsen N, Kamphuis M, de Wilde J. Preventive child health care guidelines secondary prevention child abuse (in Dutch). Bilthoven: National Institute for Public Health and the Environment; 2010.

    Google Scholar 

  17. Fleuren MAH, Van Dommelen P, Dunnink T. A systematic approach to implementing and evaluating clinical guidelines: the results of fifteen years of preventive child health care guidelines in the Netherlands. Soc Sci Med. 2015;136–137:35–43.

    Article  PubMed  Google Scholar 

  18. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318(7182):527–30.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PAC, Rubin HR. Why don’t physicians follow clinical practice guidelines?: a framework for improvement. J Am Med Assoc. 1999;282(15):1458–65.

    Article  CAS  Google Scholar 

  20. Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM. How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011;6:26.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Arts DL, Voncken AG, Medlock S, Abu-Hanna A, van Weert HCPM. Reasons for intentional guideline non-adherence: a systematic review. Int J Med Inform. 2016;89:55–62.

    Article  PubMed  Google Scholar 

  22. Konijnendijk AAJ, Boere-Boonekamp MM, Fleuren MAH, Haasnoot ME, Need A. What factors increase Dutch child health care professionals’ adherence to a national guideline on preventing child abuse and neglect? Child Abuse Negl. 2016;53:118–27.

    Article  PubMed  Google Scholar 

  23. Vandvik PO, Brandt L, Alonso-Coello P, Treweek S, Akl EA, Kristiansen A, Fog-Heen A, Agoritsas T, Montori VM, Guyatt G. Creating clinical practice guidelines we can trust, use, and share: a new era is imminent. Chest. 2013;144(2):381–9.

    Article  PubMed  Google Scholar 

  24. Latoszek-Berendsen A, Tange H, van den Herik HJ, Hasman A. From clinical practice guidelines to computer-interpretable guidelines: A literature overview. Methods Inf Med. 2010;49(6):550–70.

    Article  CAS  PubMed  Google Scholar 

  25. Black AD, Car J, Pagliai C, Anandan C, Cresswell K, Bokun T, McKinstry B, Proctor R, Majeed A, Sheikh A. The impact of eHealth on the quality and safety of health care: A systematic overview. PLoS Med. 2011;8(1):e1000387.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Dowding DW, Turley M, Garrido T. Nurses’ use of an integrated electronic health record: results of a case site analysis. Inform Health Soc Care. 2015;40(4):345–61.

    Article  PubMed  Google Scholar 

  27. McCullough JS, Casey M, Moscovice I, Prasad S. The effect of health information technology on quality in U.S. hospitals. Health Aff. 2010;29(4):647–54.

    Article  Google Scholar 

  28. Konijnendijk AAJ, Boere-Boonekamp MM, Haasnoot-Smallegange RME, Need A. A qualitative exploration of factors that facilitate and impede adherence to child abuse prevention guidelines in Dutch preventive child health care. J Eval Clin Pract. 2014;20(4):417–24.

    Article  PubMed  Google Scholar 

  29. Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–38.

    Article  CAS  PubMed  Google Scholar 

  30. Mollon B, Chong JJR, Sung M, Thabane L, Foster G. Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials. BMC Med Inform Decis Mak. 2009;9:11.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform. 2012;45(6):1202–16.

    Article  PubMed  Google Scholar 

  32. Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev. 2009;3:CD001096.

    Google Scholar 

  33. Joosse, R., Kelders, S.M., Konijnendijk, A.A.J. Supporting adherence to the guideline of secondary prevention of child abuse. The development of a digital tool for child healthcare professionals (master thesis). 2013. http://essay.utwente.nl/63517/. Accessed 25 July 2014.

    Google Scholar 

  34. Trivedi MH, Kern JK, Marcee A, Grannemann B, Kleiber B, Bettinger T, Altshuler KZ, McClelland A. Development and implementation of computerized clinical guidelines: barriers and solutions. Methods Inf Med. 2002;41(5):435–42.

    Article  CAS  PubMed  Google Scholar 

  35. Atienza AA, King AC. Community-based health intervention trials: an overview of methodological issues. Epidemiol Rev. 2002;24(1):72–9.

    Article  PubMed  Google Scholar 

  36. Wieske RCN, Nijnuis MG, Carmiggelt BC, Wagenaar-Fischer MM, Boere-Boonekamp MM. Preventive youth health care in 11 European countries: an exploratory analysis. Int J Public Health. 2012;57(3):637–41.

    Article  PubMed  Google Scholar 

  37. Dunnink, G., Lijs-Spek, W.J.G. Activiteiten Basistakenpakket Jeugdgezondheidszorg 0–19 jaar per contactmoment (in Dutch). 2008. https://www.rivm.nl/bibliotheek/rapporten/295001001.pdf. Accessed 12 Nov 2017.

    Google Scholar 

  38. GGD Twente. Twentse Gezondheidsverkenning (TGV). 2014. https://ggdtwente.buurtmonitor.nl/jive. Accessed 12 Nov 2017.

    Google Scholar 

  39. Smeeth L, Ng ESW. Intraclass correlation coefficients for cluster randomized trials in primary care: data from the MRC trial of the assessment and management of older people in the community. Control Clin Trials. 2002;23(4):409–21.

    Article  PubMed  Google Scholar 

  40. Euser S, Alink LRA, Pannebakker F, Vogels T, Bakermans-Kranenburg MJ, van Ijzendoorn MH. The prevalence of child maltreatment in the Netherlands acrosss a 5-year period. Child Abuse Negl. 2013;37(10):841–51.

    Article  PubMed  Google Scholar 

  41. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999;319(7211):670–4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. LimeSurvey Project Team / Carsten Schmitz. LimeSurvey: an open source survey tool. Hamburg; 2012. http://www.limesurvey.org.

  43. González-Izquierdo A, Woodman J, Copley L, van der Meulen J, Brandon M, Hodes D, Lecky F, Gilbert R. Variation in recording of child maltreatment in administrative records of hospital admissions for injury in England, 1997-2009. Arch Dis Child. 2010;95(11):918–25.

    Article  PubMed  Google Scholar 

  44. Woodman J, Allister J, Rafi I, De Lusignan S, Belsey J, Petersen I, Gilbert R. A simple approach to improve recording of concerns about child maltreatment in primary care records: developing a quality improvement intervention. Br J Gen Pract. 2012;62(600):e478–86.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Kruize A, Boendermaker M, Bieleman B. Monitor 2016 Huiselijk geweld Twente. 2016. https://www.breuerintraval.nl/publicatie/monitor-2016-huiselijk-geweld-twente/. Accessed 12 Nov 2017.

  46. Heckman JJ. Selection bias and self-selection. In: Eatwell J, Milgate M, Newman P, editors. Econometrics. London: Palgrave Macmillan UK; 1990. p. 201–24.

    Chapter  Google Scholar 

  47. Furnham A. Response bias, social desirability and dissimulation. Personal Individ Differ. 1986;7(3):385–400.

    Article  Google Scholar 

  48. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989;13(3):319–40.

    Article  Google Scholar 

  49. Rogers EM. Diffusion of innovations. New York: Free Press; 2003.

    Google Scholar 

  50. Venkatesh V. Determinants of perceived ease of use: integrating control, intrinsic motivation, and emotion into the technology acceptance model. Inf Syst Res. 2000;11(4):342–65.

    Article  Google Scholar 

  51. Holden RJ, Karsh BT. The technology acceptance model: its past and its future in health care. J Biomed Inform. 2010;43(1):159–72.

    Article  PubMed  Google Scholar 

  52. Schols MW, de Ruiter C, Öry FG. How do public child healthcare professionals and primary school teachers identify and handle child abuse cases? A qualitative study. BMC Public Health. 2013;13(1):807.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Thurmond VA. The point of triangulation. J Nurs Scholarsh. 2001;33(3):253–8.

    Article  CAS  PubMed  Google Scholar 

  54. Farias M, Friedman KG, Powell AJ, de Ferranti SD, Marshall AC, Brown DW, Kulik TJ. Dynamic evolution of practice guidelines: analysis of deviations from assessment and management plans. Pediatrics. 2012;130(1):93–8.

    Article  PubMed  Google Scholar 

  55. Central Committe on Research Involving human Subjects. 2015. https://english.ccmo.nl/investigators/legal-framework-for-medical-scientific-research/your-research-is-it-subject-to-the-wmo-or-not. Accessed 6 Oct 2015.

Download references

Acknowledgements

The authors would like to express their gratitude to the department of child healthcare of GGD Twente, who contributed to this study, and to Claudia van Ooijen who contributed greatly to the implementation of the tool in the computerised guideline support system.

Funding

This study was funded by the Netherlands Organization for Health Research and Development (ZonMw) (grant number 159010003).

Author information

Authors and Affiliations

  1. Department of Health Technology and Services Research, Technical Medical Centre, University of Twente, Institute for Innovation and Governance Studies, P.O. Box 217, 7500 BK, Enschede, the Netherlands

    Annemieke A. J. Konijnendijk & Magda M. Boere-Boonekamp

  2. Municipal Health Service GGD Twente, Department of Preventive Child Healthcare, P.O. Box 1400, 7500 BK, Enschede, the Netherlands

    Maria E. Haasnoot

  3. Department of Public Administration, Institute for Innovation and Governance Studies, University of Twente, P.O. Box 217, 7500 BK, Enschede, the Netherlands

    Ariana Need

Authors
  1. Annemieke A. J. Konijnendijk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Magda M. Boere-Boonekamp
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maria E. Haasnoot
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ariana Need
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AAJK: study design, ethical approval, data-collection, data analysis, data interpretation, drafting and revising manuscript. MMB-B: study design, data interpretation, reviewing manuscript. MEH: study design, anonymising record data, data interpretation, reviewing manuscript. AN: study design, data interpretation, reviewing manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Annemieke A. J. Konijnendijk.

Ethics declarations

Ethics approval and consent to participate

This research was approved by the institutional ethical committee of the Faculty of Behavioral, Management and Social sciences of the University of Twente (reference number 15028). The healthcare research Involving Human Subjects Act (also known by the Dutch acronym WMO) does not apply to this study [55].

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1

: Questionnaires for the intervention and control groups. (DOCX 36 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Konijnendijk, A.A.J., Boere-Boonekamp, M.M., Haasnoot, M.E. et al. Effects of a computerised guideline support tool on child healthcare professionals’ response to suspicions of child abuse and neglect: a community-based intervention trial. BMC Med Inform Decis Mak 19, 161 (2019). https://doi.org/10.1186/s12911-019-0884-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12911-019-0884-y

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us