Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Capturing judgement strategies in risk assessments with improved quality of clinical information: How nurses’ strategies differ from the ecological model

BMC Medical Informatics and Decision Making volume 16, Article number: 7 (2015) Cite this article

Abstract

Background

Nurses’ risk assessments of patients at risk of deterioration are sometimes suboptimal. Advances in clinical simulation mean higher quality information can be used as an alternative to traditional paper-based approaches as a means of improving judgement. This paper tests the hypothesis that nurses’ judgement strategies and policies change as the quality of information used by nurses in simulation changes.

Methods

Sixty-three student nurses and 34 experienced viewed 25 paper-case based and 25 clinically simulated scenarios, derived from real cases, and judged whether the (simulated) patient was at ‘risk’ of acute deterioration. Criteria of judgement “correctness” came from the same real cases. Information relative weights were calculated to examine judgement policies of individual nurses. Group comparisons of nurses and students under both paper and clinical simulation conditions were undertaken using non parametric statistical tests. Judgment policies were also compared to the ecological statistical model. Cumulative relative weights were calculated to assess how much information nurses used when making judgements. Receiver operating characteristic (ROC) curves were generated to examine predictive accuracy amongst the nurses.

Results

There were significant variations between nurses’ judgement policies and those optimal policies determined by the ecological model. Nurses significantly underused the cues of consciousness level, respiration rate, and systolic blood pressure than the ecological model requires. However, in clinical simulations, they tended to make appropriate use of heart rate, with non-significant difference in the relative weights of heart rate between clinical simulations and the ecological model. Experienced nurses paid substantially more attention to respiration rate in the simulated setting compared to paper cases, while students maintained a similar attentive level to this cue. This led to a non-significant difference in relative weights of respiration rate between experienced nurses and students.

Conclusions

Improving the quality of information by clinical simulations significantly impacted on nurses’ judgement policies of risk assessments. Nurses’ judgement strategies also varied with the increased years of experience. Such variations in processing clinical information may contribute to nurses’ suboptimal judgements in clinical practice. Constructing predictive models of common judgement situations, and increasing nurses’ awareness of information weightings in such models may help improve judgements made by nurses.

Peer Review reports

Background

Nurses use their judgement to identify patients at risk of deterioration in acute care. Critical care outreach systems - often nurse-led - have been implemented as a means of improving the quality of these judgements [1, 2]. Critical care outreach systems are usually initiated by track and trigger systems based on routine observations of airway adequacy, breathing and circulatory systems. Adverse observations trigger intervention and management, often in the form of expertise - such as that offered by the critical care outreach team.

In order to ensure prompt identification of patients at risk of clinical deterioration the National Institute for Health and Clinical Excellence (NICE) suggests that all hospital adult patients should receive a minimum set of physiological observations and a clear written monitoring plan at the time of admission or initial assessment [3]. These routine observations are often carried out by ward or emergency department nursing staff and they should be monitored at least every 12 h [3]. Early recognition of changes to physiological parameters is crucial to timely intervention and reducing the chances of critical events [4] and mortality. As up to 62 % of cardiac arrests are potentially avoidable [5] the evidence is that this system of identification and intervention, based largely on clinical judgement, could perform better.

Understanding the underlying mechanisms influencing information processing in nurses’ unassisted risk assessments – i.e. how nurses weigh and combine information cues to reach their judgements – would help in the design of the kinds of complex interventions (such as critical care outreach systems) likely to improve early detection and timely intervention in patients at risk of critical events or deterioration.

Critical events in patients are often preceded by changes in physiological parameters sometimes hours prior to the event [6, 7]. The National Confidential Enquiry into Patient Outcome and Death [8] suggests that approximately 66 % of patients in hospital for more than 24 h showed physiological abnormalities at least twelve hours prior to intensive care unit admission. Abnormal physiological signs are associated with patient mortality: Goldhill and McNarry [9] found a patient’s mortality risk increased with the number of physiological abnormalities (P < 0.001), being 0.7 % with no abnormalities, 4.4 % with one, 9.2 % with two and 21.3 % with three or more.

Evidence suggests that physiological deterioration is often unrecognised inadequately treated or dealt with inappropriately by healthcare professionals – including nurses [5, 10, 11]. Suboptimal care prior to critical care admission is relatively common, with at least 39 % of acute adult emergency patients being admitted to intensive care late in the clinical course of their illness [12]. Nurses’ failure to act appropriately is a major cause of suboptimal care: signs of deterioration in the 24 h prior to cardiac arrest in hospital were not acted on in 48 % of patients [5]. Aggressively intervening early for critical events such as myocardial infarction and shock can significantly reduce mortality [13, 14].

Understanding the underlying cognitive mechanisms of how nurses process clinical information to make risk assessments can help explain nurses’ suboptimal judgements.

This study aims to:

  1. i)

    describe nurses’ judgement policies in the recognition of patients at risk of acute deterioration

  2. ii)

    investigate whether their policies differ from optimal strategies derived from statistical models of the relationship between clinical information/cues and clinical outcomes (the ecology)

  3. iii)

    examine whether experienced nurses’ judgement policies differ from those of novice nurses (on the basis that clinical experience has been identified as an important factor influencing clinicians’ judgement and decision making [15, 16].

With the advent of technologies such as computerised patient simulators (e.g. Laerdal SimMan™) educators can recreate far more realistic clinical judgement situations than traditional “paper-and-pen” patients - substantially enhancing the quality of clinical information for use in simulation. We also, therefore, set out to investigate the hypothesis that nurses’ judgement policies change as the quality of information in clinical simulations also changes.

Methods

Capturing judgement policies: Cue utilization validity & ecological validity

Judgement analysis and the Lens Model of cognition (see Fig. 1), was used to investigate nurses’ judgement policies. This model characterises judgement as a relationship between a judgement and the information present in an environment (known as the ecology) and used in a judgement [17, 18]. It is based largely on the principles of probabilistic functionalism put forward by Egon Brunswik [19–22].

Fig. 1
figure 1

The Lens Model

Full size image

Using the Lens Model (Fig. 1) to understand judgements requires explication of three important concepts: ecological validity, cue utilisation validity, and ‘achievement’ [23]. Ecological validity refers to the correlations between proximal cues and an ecological criterion [20] (such as disease classification or level of risk). Cue utilisation validity refers to correlations between proximal cues and an individual’s judgements. Achievement refers to the correlations between the values of an ecological criterion and the values of an individual’s judgement.

In the Lens Model the left side represents the relationship (i.e. the ecological validity) between proximal cues and the ecological criterion i.e. how clinical information is correlated with an actual clinical state. The right side of the Lens Model represents the relationship (i.e. cue utilisation validity) between available cues and subjects’ judgements i.e. the importance clinicians attach to clinical information cues. Clinicians’ judgements may (and crucially, may not) be similar to the weights in the true ecology. Achievement is captured by the correlation (i.e. accuracy) between subjects’ judgements and the ecological criterion. In this paper we focus on cue utilisation validity and compare it to ecological validity: how participants use the cues in their judgements as compared to the weights in a statistical model of the judgement ecology.

Clinical scenarios and data collection

We used five cues as recommended by NICE [3] to construct the 25 scenarios: systolic blood pressure, heart rate, respiratory rate, temperature, and levels of consciousness. The 25 clinical scenarios were randomly sampled from a large data set of patient case series (n = 673) [24]. Judgement ecological criteria (reference standards) were derived from the same set of patient case records: the patient being simulated was classified as at risk if the patient case was admitted to intensive care units, had cardiopulmonary resuscitation or died.

All the clinical scenarios were presented as paper cases in a booklet of clinical vignettes (see the Additional file 1 where the patient name used is a pseudonym). Natural units (such as mmHg for blood pressure and beats per minute for Heart rate) that are routinely used in the current practice were used when presenting these cues. A patient simulator (Laerdal SimMan) and bedside vital signs monitor were used to simulate the same clinical scenarios presented in the booklet of clinical vignettes. Scenarios and clinical simulations were approved by a critical care specialist nurse with more than 12 years specialist nursing experience. Nurses were asked to assess risk of a critical event independently, for each paper based scenario and then again on a data collection sheet in clinical simulations. They were asked to complete all 25 scenarios on paper cases and subsequently complete all 25 scenarios on clinical simulations. Nurses were instructed not to discuss these scenarios with each other and recorded their assessments into the data collection sheet independently. Nurses were solely asked to assess these scenarios to detect whether the simulated patient case was at risk of acute deterioration, but they were not required to take a medical intervention in response to these scenarios where the patient case was at risk of acute deterioration.

Ethical approval

Ethical approval was obtained from the Health Sciences Research Governance Committee of the University of York, UK. All participants were given the same information about the study. All participants completed an informed consent document.

Data analysis

Relative weights of validity coefficients

Relative cue weights for individual nurses in this study were derived from the cue utilisation validity (the cue-judgement correlation) and the ecological validity (the cue-criterion correlation). Relative weights can be derived from either regression models or cue validities [25]; we derived the relative cue weights for each participant from cue validities: the cue utilisation validity (cue-judgement correlation) and the ecological validity (cue-criterion correlation). Cue-judgement and cue-criterion correlations index a cue’s importance to the prediction of judgements or criteria (the ecology) [20, 26–28]. The correlations between cues and judgements indicate the emphasis judges placed on these cues; the predictive ability increases when the correlation between each cue and its dependent variables increases.

For each participant’s model the relative weights of cues were generated from the validity coefficients by normalising their absolute values to 1:

$$ r{w}_i=\frac{\left|{r}_{Ys. Xi}\right|}{{\displaystyle \sum_{i=1}^K|{r}_{Ys. Xi}|}} $$

To investigate the relative weight of the categorical variable of consciousness level, use of a single cue weight represents the judge’s overall emphasis on the information contained in the categorical cue [25]. In the analysis, the categorical cue of consciousness level was therefore put in the model together with other continuous cues to identify the single overall effect.

Cumulative relative weights were calculated from the relative weights of cue utilisation validity to assess how much information participants used in making their judgements. The cumulative relative weights of the third most important cue were taken as the optimal cut-off measure to evaluate the amount of information participants have used. A statistical comparison of cumulative relative weights between participants and the ecological model was also made.

Where appropriate non-parametric tests were used to detect the statistical significance of differences in relative weights between different groups since the data did not follow a normal distribution. For example, the Wilcoxon rank-sum test was used to test the significance of the difference between student and experienced nurse groups. The Wilcoxon matched-pairs signed-ranks test was used to test the significance of the difference between paper case and clinical simulations. A level of p < 0.05 was used as a cut-off for statistical significance.

Analysis of ROC curves

To identify the model’s predictive accuracy for participants’ judgements, ROC curves were generated from the aggregate models of paper cases and clinical simulations. ROC curves were obtained by plotting the sensitivity (true positive rate) against 1− specifity (false positive rate) [29]. The predictive accuracy of alternative logistic models used were assessed by comparing the areas under the corresponding ROC curves (AUC) [30]. The higher the AUC the greater the predictive performance of the model (an AUC of 0.5 would be “chance” or the equivalent of flipping an unbiased coin).

Results

Participants

Sixty-three students and 34 experienced nurses took part. The nurses had an average of 12 years (standard deviation (SD) 10.0) of clinical experience; the majority (n = 27, 81 %) were educated to diploma or first degree level. The mean age of experienced nurses was 36.6 years (SD 9.9), while the mean age of student nurses was 29.1 years (SD 9.0). 85 % of experienced nurses were female and 89 % of student nurses were female. The experienced nurses were recruited from the ward & critical care registered nurse population in hospitals in North Yorkshire. The majority of students were 2nd and 3rd year undergraduate nurses and registered for a diploma in nursing.

Relative weights of Cue utilisation validity & ecological validity

Experienced nurses vs. Students

In paper cases experienced nurses paid more attention to consciousness level (median (Mdn) 0.259) than students (Mdn 0.240), z = −2.39, P = 0.02 and more attention to systolic blood pressure (Mdn 0.065) than students (Mdn 0.054), z = −1.97, P = 0.048. The experienced nurses paid less attention to respiration rate (Mdn 0.247) than students (Med 0.278), z = 2.80, P = 0.005. There was no significant difference in the relative weights given to heart rate and temperature by experienced nurses and students.

In clinical simulation settings experienced nurses paid more attention to consciousness level (Mdn 0.253) than students (Mdn 0.228), z = −3.22, P = 0.001 and systolic blood pressure (Mdn 0.083) than students (Mdn 0.050), z = −2.50, P = 0.01. The experienced nurses paid less attention to temperature (Mdn 0.148) than students (Mdn 0.167), z = 2.41, P = 0.02. There were no differences in relative weightings given to heart rate and respiration rate by experienced and student nurses.

Paper cases vs. the ecology model

In paper cases participants underused the consciousness level cue (Mdn 0.245); the ecological model suggests a figure of (Mdn) 0.301 would be more appropriate (z = −10.14, P < 0.001). They also underused respiration rate (Mdn 0.275; ecological model requires (Mdn 0.320), z = −7.19, P < 0.001), and systolic blood pressure (Mdn 0.057; ecological model (Mdn 0.079), z = −6.22, P < 0.001). In contrast participants over-relied on heart rate (Mdn 0.265; ecological model (Mdn 0.248), z = 3.43, P < 0.001), and over-relied on temperature (Mdn 0.149; ecological model (Mdn 0.052), z = 10.15, P < 0.001).

Physical simulation vs. the ecology model

In clinical simulation settings participants underused the consciousness level cue (Mdn 0.236; ecological model (Mdn 0.301), z = −8.42, P < 0.001); and respiration rate (Mdn 0.280; ecological model (Mdn 0.320), z = −4.91, P < 0.001), and systolic blood pressure (Mdn 0.058; ecological model (Mdn 0.079), z = 3.74, P < 0.001). Participants over-relied on temperature (Mdn 0.158) in clinical simulations (ecological model, (Mdn 0.052), z = 8.54, P < 0.001). However, there was no significant difference in the relative weights of heart rate between clinical simulation and the ecological model. Because the relative weight in the ecological model provides an optimal standard of how the participant should pay an attention to a particular cue, this non-significant difference in relative weights of heart rate between the two models indicated that the clinical simulation setting was associated with appropriate use of heart rate information.

Analysis of cumulative relative weights

The results showed that participants’ cumulative relative weights on the third most important cue (Mdn 0.795) in paper cases were lower than the equivalent cumulative relative weights (0.869) in the ecology model z = −10.14, P < 0.001. Similarly, participants’ cumulative relative weights on the third most important cue (Mdn 0.788) in clinical simulations were lower than the equivalent cumulative relative weights (0.869) in the ecology, z = −8.08, P < 0.001.

Analysis of ROC curves

ROC curves were plotted for hierarchical logistic regression models for participants at the aggregate level in paper cases (Fig. 2). Each cue was sequentially entered into the models based on the ranking of the binary correlations between each cue and the dependent variable of judgements in paper cases. The AUC area with 95 % confidence interval (CI) for each model was calculated. Table 1 illustrates the model performance of the hierarchical logistic regression of judgement in paper cases.

Fig. 2
figure 2

ROC curves of hierarchical models in paper cases

Full size image
Table 1 The model performance and the prediction of judgement in paper cases
Full size table

In the paper cases the AUC curves showed that heart rate, respiration rate and consciousness level were the three major predictors of 97 participants’ judgements. The AUC area for each model was consistent with the cumulative R-squared of each hierarchical model. Temperature and systolic blood pressure information was of no added value in predicting participants’ judgements in paper cases.

ROC Curves were also plotted for hierarchical logistic regression models for the same 97 participants at the aggregate level in clinical simulations (Fig. 3). The cues were sequentially entered into the models based on the ranking of the binary correlation between each cue and the dependent variable of judgements in clinical simulations. Table 2 shows the model performance of the hierarchical logistic regression of judgement in clinical simulations.

Fig. 3
figure 3

ROC curves of hierarchical models in clinical simulation

Full size image
Table 2 The model performance and the prediction of judgement in clinical simulations
Full size table

In clinical simulations the ROC curves showed respiration rate, heart rate, and consciousness level were the three main predictors of participants’ judgements. It should be noted that the cumulative R-squared of hierarchical models showed that temperature may still be an additional contributor in predicting judgements of clinical simulations. However, systolic blood pressure was of no added value in predicting participants’ judgements in clinical simulation.

Discussion

There are wide variations in individual nurses’ use of information cues (based on relative cue weights of cue utilisation validities) in paper cases and clinical simulations; a finding in line with previous research [31]. Such variations reflect the substantial differences in cue usage among individual nurses when faced with different judgement tasks.

Judgement policies compared with the ecological model

Despite such wide variations, nurses both under-values some cues (consciousness level, respiration rate and systolic blood pressure) and put too much weight on others (heart rate and temperature) when faced with (traditional) paper based scenarios. Similar patterns were observed in the more “advanced” simulated environment of the clinical simulator (with the exception that heart rate was weighted more appropriately). Both experienced nurses and students both used more cues than needed to make appropriate judgements; this matters. Assigning similar weights to those in the ecology will improve judgement performance. For example less relevant information can reasonably be ignored [32]. If an irrelevant cue is weighted, it significantly lowers the utilisation of other relevant cues in a particular judgement, thereby resulting in an error. Providing nurses with optimal weights using models of the ecology may minimize their policy variations and improve judgement achievement.

Judgement policies in experienced nurses and students

Cue utilisation relative weights revealed experienced nurses and students differs substantially on the cues used to make risk assessments: experienced nurses rely more on the cues of consciousness level and systolic blood pressure for risk ssessment judgements than students. These differences in relative weighting could be an important source of inconsistencies and variations in judgements between experienced nurses and nurse students.

Judgement policies in paper cases and clinical simulation

Generally, similar patterns of relative weights of cue utilisation validity were observed in both paper cases and clinical simulations. But setting impacted on heart rate (experienced nurses over relied on it for paper cases but used it more appropriately in clinical simulations) suggesting that clinical simulation may foster a more appropriate use of at least some information. Moreover, nurses used more available information in clinical simulations than in paper cases. It is reasonable then to argue that different simulation approaches lead to variations in nurses’ judgement policies. This is in line with (or extends the logic of) studies [33–35], suggesting the format of information presentation (e.g. pictorial information is substituted for written description) significantly affects the amount of information the subjects would use. Importantly, these findings reveal that, under more natural settings where the quality of clinical information is improved, nurses use strategies that are markedly different from those elicited by paper cases.

Differences in nurse’ judgement strategies between paper cases and clinical simulations can be explained theoretically using Cognitive Continuum Theory [36]. According to this theory, pictorial cues in clinical simulations may induce more intuition in judgements, whilst more “abstract” (i.e. less visual) quantitative cues in paper cases will promote a more analytic approach. Therefore, changes in the format of information presentation may result in marked differences in judgement strategies between paper cases and clinical simulations.

The ROC curve analyses illustrate that three cues (respiration rate consciousness level and heart rate) were primarily used to reach judgements (regardless of simulation approach). This implies that nurses place more importance on these three cues in risk assessment judgements than the other two (systolic blood pressure and temperature). These findings suggest that nurses may “miss” information contained in an assessment of risk in critically ill patients.

Strengths and limitations

In this study we used real patient cases to construct clinical scenarios which substantially enhanced the representativeness of clinical scenarios. Despite this strength, it should be recognised that cue intercorrelations are a feature of ‘vicarious functioning’ [37] in real world judgement tasks. The high levels of cue intercorrelations challenge the analysis of relative importance of cues and a large number of judgement tasks is required [38, 39]. Particularly, large numbers of judgement tasks are practically limiting: participants’ boredom or impatience as a result of judging an excessive number of tasks may influence their judgement processes [40]. In this study we derived cue relative weights using cue utilisation validity (cue-judgement correlation) and ecological validity (cue-criterion correlation), as advocated by Brunswik [20] and Hammond [26] in their early work. These relative weights of validity coefficients are able to index the independent contribution of each cue to the predictions of judgements and the ecology. This useful cue weighting approach provides a more holistic picture of how nurses value each cue compared to the ecological model.

Implication for research

Cognitive feedback, a type of feedback describing the relations between symptoms/signs and outcomes, essentially captures the probabilistic nature of tasks and the inherent uncertainty of environment [41, 42]. This type of feedback could help improve nurses’ judgement performance. For a typical task in risk assessments, not only are nurses required to classify patients as at “risk” or “not at risk”, but also to learn the relationships between symptoms and disease outcomes. These relationships (depicted as probabilistic functionalism [26, 43]) play a significant role to help nurses gain such abilities in probabilistic inference. Providing cognitive feedbacks with task information has proved to be useful for improving judgement performance in doctors [44–47]. The potential of using cognitive feedbacks to improve nurses’ judgements should therefore be tested for future research.

Conclusions

The findings from this study suggest that a mismatch between nurses’ judgement strategies and a pertinent ecological model may explain suboptimal judgement performance. Our findings highlight the importance of appropriate attention being paid to patients exhibiting physiological abnormalities, and correct recognition of patients at risk of acute deterioration. When making risk assessments in practice, nurses should pay more attention to important cues such as respiration rate, consciousness level and systolic blood pressure. Clinical simulations recreating real patient cases have advantages over paper based traditional approaches to simulation. These simulations and the awareness they provide can allow nurses to refine their judgement behaviours with opportunities for reflection and correction. The report by the National Patient Safety Agency [48] has revealed that insufficient training to understand the relevance of observations is one common factor contributing to patient deterioration incidents not being recognised or acted upon. Approaches such as cognitive feedback could enhance understanding of the relationship between clinical information and ecological criteria (for example, physiological signs and symptoms and true underlying risk of a critical event); it may also help nurses appreciate more about the nature of uncertainty in the probabilistic relationships that make up the provision of healthcare.

Abbreviations

ROC:

Receiver operating characteristic

NICE:

National Institute for Health and Clinical Excellence

AUC:

Areas under the corresponding ROC curves

CI:

Confidence interval

HR:

Heart Rate

RR:

Respiration rate

BP:

Blood pressure

References

  1. Intensive Care Society. Guidelines for the introduction of Outreach Services. London: Intensive Care Society; 2002. p. 1–18.

    Google Scholar 

  2. Department of Health. Clinical indicators for critical care outreach services. London: Department of Health; 2007. p. 1–11.

    Google Scholar 

  3. National Institute for Health Clinical Excellence: Acutely ill patients in hospital: recognition of and response to acute illness in adults in hospital. In., vol. 50: National Institute for Health and Clinical Excellence (NICE): clinical guideline; 2007: 1–107.

  4. McArthur-Rouse F. Critical care outreach services and early warning scoring systems: a review of the literature. J Adv Nurs. 2001;36(5):696–704.

    Article  CAS  PubMed  Google Scholar 

  5. Hodgetts TJ, Kenward G, Vlackonikolis I, Payne S, Castle N, Crouch R, et al. Incidence, location and reasons for avoidable in-hospital cardiac arrest in a district general hospital. Resuscitation. 2002;54(2):115–23.

    Article  PubMed  Google Scholar 

  6. Schein RM, Hazday N, Pena M, Rubens BH, Sprung CL. Clinical antecedents to in-hospital-cardiopulmonary arrest. Chest. 1990;98:1388–92.

    Article  CAS  PubMed  Google Scholar 

  7. Buist MD, Jarmolowski E, Burton PR, Bernard SA, Waxman BP, Anderson J. Recognising clinical instability in hospital patients before cardiac arrest or unplanned admission to intensive care. A pilot study in a tertiary-care hospital. Med J Aust. 1999;171:22–5.

    CAS  PubMed  Google Scholar 

  8. Ncepod. An acute problem? A report of the National Confidential Enquiry into Patient Outcome and Death (NCEPOD). London: NCEPOD; 2005.

    Google Scholar 

  9. Goldhill DR, McNarry AF. Physiological abnormalities in early warning scores are related to mortality in adult inpatients. Br J Anaesth. 2004;882–884.

  10. Franklin C, Mathew J. Developing strategies to prevent inhospital cardiac arrest: analyzing responses of physicians and nurses in the hours before the event. Crit Care Med. 1994;22:244–7.

    Article  CAS  PubMed  Google Scholar 

  11. Rich K. Inhospital cardiac arrest: pre-event variables and nursing response. Clin Nurse Spec. 1999;13:147–53.

    Article  CAS  PubMed  Google Scholar 

  12. McQuillan P, Pilkingons S, Allan A, Taylor B, Short A, Morgan G, et al. Confidential inquiry into quality of care before admission to intensive care. Br Med J. 1998;316:1853–8.

    Article  CAS  Google Scholar 

  13. Lamfers EJP, Hooghoudt TE, Schut A, de Boo T, Verheugt FW. Time from symptom onset to treatment and outcome in prehospital thrombolysis for acute ST-elevation myocardial infarction. Neth Heart J. 2002;10:449–54.

    CAS  PubMed  PubMed Central  Google Scholar 

  14. Rivers E, Nguryen B, Havstad S, Ressler J, Muzzin A, Knoblich B, et al. Early Goal-Directed Therapy in the Treatment of Severe Sepsis and Septic Shock. N Engl J Med. 2001;345(19):1368–77.

    Article  CAS  PubMed  Google Scholar 

  15. Benner P. From novice to expert: excellence and power in clinical nursing practice. California: Addison-Wesley Publishing Company; 1984.

    Google Scholar 

  16. Benner P, Tanner C, Chesla C. Expertise in nursing practice: caring, clinical judgment, and ethics. New York: Springer Publication; 1996.

    Google Scholar 

  17. Hammond KR, Adelman L. Science, values, and human judgment. Science. 1976;194:389–96.

    Article  CAS  PubMed  Google Scholar 

  18. Hammond KR, Stewart TR, Brehmer B, Steinmann D. Social judgment theory. In: Kaplan MF, Schwartz S, editors. Human judgment and decision processes. New York: Academic; 1975. p. 271–312.

    Chapter  Google Scholar 

  19. Brunswik E. In defense of probabilistic functionalism: a reply. Psychol Bull. 1955;62(3):236–42.

    CAS  Google Scholar 

  20. Brunswik E. Representative design and probabilistic theory in a functional psychology. Psychol Review. 1955;62(3):193–217.

    Article  CAS  Google Scholar 

  21. Brunswik E. Remarks on functionalism in perception. In: Hammond KR, Stewart TR, editors. Essential Brunswik: beginnings, explications, applications. Oxford: Oxford University Press; 1949. p. 127–33.

    Google Scholar 

  22. Brunswik E. Organismic achievement and environmental probability. Psychol Review. 1943;50:255–72.

    Article  Google Scholar 

  23. Cooksey RW. The methodology of social judgement theory. Think Reason. 1996;2:141–73.

    Article  Google Scholar 

  24. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM. 2001;94(10):521–6.

    Article  CAS  PubMed  Google Scholar 

  25. Cooksey RW. Judgment analysis: theory, methods, and applications. California: Academic; 1996.

    Google Scholar 

  26. Hammond KR. Probabilistic functioning and the clinical method. Psychol Rev. 1955;62:255–62.

    Article  CAS  PubMed  Google Scholar 

  27. Browne BA, Gillis JS. Evaluating the quality of instruction in art: a social judgement analysis. Psychol Rep. 1982;50:955–62.

    Article  Google Scholar 

  28. Hagafors R, Brehmer B. Does having to justify one’s judgements change the nature of the judgement process? Organ Behav Hum Perform. 1983;31:223–32.

    Article  Google Scholar 

  29. Altman DG, Bland JM. Statistics Notes: Diagnostic tests 3: receiver operating characteristic plots. BMJ. 1994;309:188.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Cleves MA. From the help desk: comparing areas under receiver operating characteristic curves from two or more probit or logit models. Stata J. 2002;2(3):301–13.

    Google Scholar 

  31. \Brehmer A, Brehmer B. What have we learned about human judgement from thirty years of policy capturing? In: Brehmer B, Joyce CRB, editors. Human judgement: the SJT view. Amsterdam: Elsevier; 1988. p. 75–114.

    Chapter  Google Scholar 

  32. Gaeth GJ, Shanteau J. Reducing the influence of irrelevant information on experienced decision makers. Organ Behav Hum Decis Process. 1984;33:263–82.

    Article  Google Scholar 

  33. Chaput de Saintonge DM, Hathaway NR. Antibiotic use in otitis media: patient simulations as an aid to audit. BMJ. 1981;283:883–4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Lamond D, Crow R, Chase J, Doggen K, Swinkels M. Information sources used in decision making: considerations for simulation development. Int J Nurs Stud. 1996;33(1):47–57.

    Article  CAS  PubMed  Google Scholar 

  35. Phelps RH, Shanteau J. Livestock judges: how much information can an expert use? Organ Behav Hum Decis Process. 1978;21:209–19.

    Article  Google Scholar 

  36. Hamm RM, Dowie J, Elstein A. Clinical intuition and clinical analysis: expertise and the cognitive continuum. In: Professional Judgment: A Reader in Clinical Decision Making. Cambridge: Cambridge University Press; 1998. p. 78–105.

    Google Scholar 

  37. Dhami M: Vicarious functioning and representative design. In., vol. 17 The Brunswik Society Newsletter 1999: 1–3.

  38. Mumpower JL, Stewart TR. Expert judgment and expert disagreement. Think Reason. 1996;2:191–211.

    Article  Google Scholar 

  39. Stewart TR. Judgment analysis: procedures. In: Brehmer B, Joyce CRB, editors. Human judgment: the SJT view. Oxford: North-Holland; 1988. p.41–74.

  40. Hamm R: Cue by hypothesis interactions in descriptive modeling of unconscious use of multiple intuitive judgment strategies. In: Intuition in Judgment and Decision Making. Edited by Plessner H, Betsch C, Betsch T. New York: Taylor & Francis Group; 2008. 55–70.

  41. Balzer WK, Doherty ME, O’Connor R. Effects of cognitive feedback on performance. Psychol Bull. 1989;106:410–33.

    Article  Google Scholar 

  42. Hammond KR, Summers DA. Cognitive control. Psychol Rev. 1972;79(1):58–67.

    Article  Google Scholar 

  43. Hammond KR. Probabilistic functionalism: Egon Brunswik's integration of the history, theory, and method of psychology. In: Hammond KR, editor. The psychology of Egon Brunswik. New York: Holt, Rinehart and Winston, INC; 1966. p. 15–80.

    Google Scholar 

  44. Stalsby Lundborg C, Wahstrom R, Oke T, Tomson G. Influencing prescribing for urinary tract infections and asthma in primary care in Sweden - a randomized controlled trial of an interactive educational intervention. J Clin Epidem. 1999;52:801–12.

    Article  Google Scholar 

  45. Veninga CCM, Lagerlov P, Wahlstrom R, Muskova M, Denig P, Gerkhof H, et al. Evaluating an educational intervention to improve the treatment of asthma in four European countries. Am J Resp Crit Care Med. 1999;160:1254–62.

    Article  CAS  PubMed  Google Scholar 

  46. Wigton RS, Patil KD, Hoellerich VL. The effect of feedback in learning clinical diagnosis. J Med Education. 1986;61:816–22.

    CAS  Google Scholar 

  47. Wigton RB. Application of judgment analysis and cognitive feedback to medicine. In: Brehmer B, Joyce CRB, editors. Human judgment: the SJT view. Oxford: North-Holland; 1988. p. 227–46.

    Chapter  Google Scholar 

  48. National Patient Safety Agency: Recognising and responding appropriately to early signs of deterioration in hospitalised patients. In., vol. 5. London: National Patient Safety Agency; 2007. p. 1–43.

Download references

Acknowledgment

This article presents independent research funded in part by the National Institute for Health Research (NIHR) through the Leeds York Bradford Collaboration for 19 Leadership in Applied Health Research and Care. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

Author information

Authors and Affiliations

  1. Centre for Reviews and Dissemination, University of York, York, UK

    Huiqin Yang

  2. School of Healthcare, Faculty of Medicine and Health, University of Leeds, Leeds, UK

    Carl Thompson

Authors
  1. Huiqin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carl Thompson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carl Thompson.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

HY and CT were responsible for the study conception and design. HY performed the data collection. HY performed the data analysis. HY was responsible for drafting the manuscript. HY and CT made critical revisions to the paper for important intellectual content. HY provided statistical expertise. All authors read and approved the final manuscript.

Additional file

Additional file 1:

Clinical background. (DOCX 18 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., Thompson, C. Capturing judgement strategies in risk assessments with improved quality of clinical information: How nurses’ strategies differ from the ecological model. BMC Med Inform Decis Mak 16, 7 (2015). https://doi.org/10.1186/s12911-016-0243-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12911-016-0243-1

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us