Table 2 I-MeDeSA [18] items perceived to be ambiguous leading to differences in interpretation among reviewers

(2i) Are different types of alerts meaningfully grouped (e.g., by the severity of the alert, where all Level 1 alerts are placed together, or by medication order, where alerts related to a specific medication order are grouped together)?

(3iii) Is the font used to display the textual message appropriate for the user to read the alert easily (e.g., a mixture of upper and lower-case lettering is easier to read than upper case only)?

(4iii) Are signal words appropriately assigned to each existing level of alert (eg, ‘Warning’ would appropriately be assigned to a Level 1 alert and not a Level 3 alert)? ‘Note’ would appropriately be assigned to a Level 3 alerts, and not a Level 1 alert.

(5ii) Is color minimally used to focus the attention of the user? Excessive coloring used on the screen can create noise and distract the user. Therefore, colors should be kept to fewer than 10.

(6i) Are the different severities of alerts easily distinguishable from one another? For example, do major alerts possess visual characteristics that are distinctly different from minor alerts? The use of a signal word to identify the severity of an alert is not considered to be a visual characteristic.