Skip to main content

Table 2 Illustrative example of comparing two diagnostic instruments (Note: At Step 1, Numbers in agreement sections in each table are expressed in boldface)

From: Measuring agreement between healthcare survey instruments using mutual information

Procedure Scenario 1 Scenario 2 Scenario 3
Data (\( {x}_{11} \), \( {x}_{12} \), \( {x}_{21} \), \( {x}_{22} \)) (10, 5, 5, 20) (5, 10, 20, 5) (5, 10, 5, 20)
Step 1
Contingency table
10
5
5
20
5
20
10
5
5
5
10
20
Step 2 \( {I_{agreement}}_{+\left(\frac{20}{40}\right){ \log}_2\frac{\left(\frac{20}{40}\right)}{\left(\frac{25}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{10}{40}\right){ \log}_2\frac{\left(\frac{10}{40}\right)}{\frac{(15)}{40}\left(\frac{15}{40}\right)}} \) \( {I_{agreement}}_{+\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\left(\frac{15}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\frac{(15)}{40}\left(\frac{25}{40}\right)}} \) \( {I_{agreement}}_{+\left(\frac{20}{40}\right){ \log}_2\frac{\left(\frac{20}{40}\right)}{\left(\frac{30}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\frac{(15)}{40}\left(\frac{10}{40}\right)}} \)
Mutual information
\( {I_{disagreement}}_{+\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\left(\frac{15}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\frac{(15)}{40}\left(\frac{25}{40}\right)}} \) \( {I_{disagreement}}_{+\left(\frac{20}{40}\right){ \log}_2\frac{\left(\frac{20}{40}\right)}{\left(\frac{25}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{10}{40}\right){ \log}_2\frac{\left(\frac{10}{40}\right)}{\frac{(15)}{40}\left(\frac{15}{40}\right)}} \) \( {I_{disagreement}}_{+\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\left(\frac{10}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{10}{40}\right){ \log}_2\frac{\left(\frac{10}{40}\right)}{\frac{(15)}{40}\left(\frac{30}{40}\right)}} \)
Step 3
Significance
Mutual information = 0.386-0.227 = 0.159
From (Eq. 6), 2*40*ln2*0.159 = 8.809
Highly significant (with p < 0.001)
Mutual information = -0.227 + 0.386 = 0.159
From (Eq. 6), 2*40*ln2*0.159 = 8.809
Highly significant (with p < 0.001)
Mutual information = 0.098-0.083 = 0.015
From (Eq. 6), 2*40*ln2*0.015 = 0.871
Less significant (with p = 0.35)
Step 4
Local mutual information
I agreement  > I disagreement
and highly significant mutual information. Thus, agreement
I agreement  < I disagreement
and highly significant mutual information. Thus, disagreement
I agreement  > I disagreement
but very low mutual information observed. Thus, inconclusive