Table 1 Overview of methods

Questionnaire

Quantitative data

Assess users’ perceptions towards the implemented CDSS (R1)

To investigate user interactions with the system and possible relationships between perceptions and use (R2)

The UTAUT model (I1)

The literature review of factors related to implementing and using CDSSs (I2)

52 doctors and nurses (8 doctors and 44 nurses)

52 questionnaires distributed and answered (100% response rate)

32 construct derived questions (1 open-ended)

After three months of system use

System usage data

Quantitative data

To assess the actual use of the system (R3) and to qualitatively analyze system usage behavior (R4)

Also to help identify the most frequent users of the system/interview subjects

Data from the system logs

All of the 81 users Central functionalities:

User login time (R3) and frequency (R3, R4)

User data/documents retrieval requests (R4)

Client workstation location (R4)

Performed for the testing and evaluation period of four months

Semi-structured interviews

Qualitative data

Same as R1 and R2, with the main emphasis being on gathering in-depth information

Same as I1 and I2, to inform construction of the interview guide

Analysis of the system logs to identify the most frequent users

14 individual interviews:

6 doctors

6 nurses

A nurse and a doctor holding leadership positions

Total, 609 min

Average 43.5 min

Shortest/longest 30/70 min

87 construct relevant questions (derived from 6 and 7)

After two months of system use

Ethnographic observations

Qualitative data

Same as R1 and R2

Facilitate an understanding of doctors and nurses everyday work situation, work practices, and perspective (R5)

Understanding the daily use of patient records in the EHR system (R6)

First-hand observation of doctors and nurses using the CDSS

Changes in work routines due to CDSS

Inform the questionnaire and interview guide (R7)

Observation of doctors and nurses in the Anesthesia and ICU department and in the anesthesiology out-patients’ clinic

119 h

28 h of user training

59 h in the Anesthesia and ICU department

32 h in the anesthesiology out-patients’ clinic

Performed during the four months system testing and evaluation period

User logbooks

Qualitative data

Same as R1, R2, and R5

Data from the user logbooks available at the client workstation locations

17 logbook entries

Performed during the four months system testing and evaluation period

Document review

Qualitative data

Same as R6 and R7

Data from the hospital trust’s enterprise EHR system

22,000 EHR documents

Performed during the system design and implementation phase