Skip to main content

Table 2 Illustrative example of comparing two diagnostic instruments (Note: At Step 1, Numbers in agreement sections in each table are expressed in boldface)

From: Measuring agreement between healthcare survey instruments using mutual information

Procedure

Scenario 1

Scenario 2

Scenario 3

Data (\( {x}_{11} \), \( {x}_{12} \), \( {x}_{21} \), \( {x}_{22} \))

(10, 5, 5, 20)

(5, 10, 20, 5)

(5, 10, 5, 20)

Step 1

Contingency table

10

5

5

20

5

20

10

5

5

5

10

20

Step 2

\( {I_{agreement}}_{+\left(\frac{20}{40}\right){ \log}_2\frac{\left(\frac{20}{40}\right)}{\left(\frac{25}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{10}{40}\right){ \log}_2\frac{\left(\frac{10}{40}\right)}{\frac{(15)}{40}\left(\frac{15}{40}\right)}} \)

\( {I_{agreement}}_{+\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\left(\frac{15}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\frac{(15)}{40}\left(\frac{25}{40}\right)}} \)

\( {I_{agreement}}_{+\left(\frac{20}{40}\right){ \log}_2\frac{\left(\frac{20}{40}\right)}{\left(\frac{30}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\frac{(15)}{40}\left(\frac{10}{40}\right)}} \)

Mutual information

\( {I_{disagreement}}_{+\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\left(\frac{15}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\frac{(15)}{40}\left(\frac{25}{40}\right)}} \)

\( {I_{disagreement}}_{+\left(\frac{20}{40}\right){ \log}_2\frac{\left(\frac{20}{40}\right)}{\left(\frac{25}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{10}{40}\right){ \log}_2\frac{\left(\frac{10}{40}\right)}{\frac{(15)}{40}\left(\frac{15}{40}\right)}} \)

\( {I_{disagreement}}_{+\left(\frac{5}{40}\right){ \log}_2\frac{\left(\frac{5}{40}\right)}{\left(\frac{10}{40}\right)\left(\frac{25}{40}\right)}}^{=\left(\frac{10}{40}\right){ \log}_2\frac{\left(\frac{10}{40}\right)}{\frac{(15)}{40}\left(\frac{30}{40}\right)}} \)

Step 3

Significance

Mutual information = 0.386-0.227 = 0.159

From (Eq. 6), 2*40*ln2*0.159 = 8.809

Highly significant (with p < 0.001)

Mutual information = -0.227 + 0.386 = 0.159

From (Eq. 6), 2*40*ln2*0.159 = 8.809

Highly significant (with p < 0.001)

Mutual information = 0.098-0.083 = 0.015

From (Eq. 6), 2*40*ln2*0.015 = 0.871

Less significant (with p = 0.35)

Step 4

Local mutual information

I agreement  > I disagreement

and highly significant mutual information. Thus, agreement

I agreement  < I disagreement

and highly significant mutual information. Thus, disagreement

I agreement  > I disagreement

but very low mutual information observed. Thus, inconclusive