The interpretation of systematic reviews with meta-analyses: an objective or subjective process?

Table 1 Decisions of the 8 reviewers based on the 6 systematic reviews with meta-analyses presented to them.

# RCTS	1	1–3	1–5	1–10	1–20	1–23
N	111	415	597	3685	63047	69505
Fixed OR	N/a	0.40 (0.19–0.83)	0.40 (0.28–0.61)	0.64 (0.52–0.79)	1.02 (0.96–1.08)	1.01 (0.96–1.07)
Rand OR	N/a	0.40 (0.18–0.86)	0.38 (0.21–0.66)	0.66 (0.53–0.81)	0.65 (0.48–0.87)	0.75 (0.61–0.92)
I²	N/a	0%	0%	21%	59%	59%
I believe magnesium has now been shown to be beneficial for patients during the post-MI period
(C)	Disagree	Unsure	Unsure	Disagree	Disagree	Disagree
(C)	Strongly Disagree	Unsure	Unsure	Unsure	Disagree	Strongly Disagree
(P)	Strongly Disagree	Agree	Agree	Agree	Agree	Agree
(P)	Agree	Agree	Strongly Agree	Strongly Agree	Strongly Agree	Agree
(P)	Strongly Disagree	Strongly Disagree	Disagree	Unsure	Agree	Agree
(P)	Unsure	Agree	Agree	Agree	Agree	Agree
(NP)	Strongly Disagree	Disagree	Disagree	Agree	Disagree	Strongly Disagree
(N)	Unsure	Agree	Agree	Agree	Agree	Agree
I believe magnesium will eventually be shown to be beneficial for patients during the post-MI period
(C)	Agree	Agree	Agree	Agree	Unsure	Disagree
(C)	Unsure	Unsure	Unsure	Unsure	Strongly Disagree	Strongly Disagree
(P)	Unsure	Agree	Agree	Strongly Agree	Agree	Agree
(P)	Agree	Strongly Agree	Strongly Agree	Strongly Agree	Strongly Agree	Agree
(P)	Strongly Disagree	Strongly Disagree	Disagree	Unsure	Agree	Agree
(P)	Agree	Agree	Agree	Agree	Agree	Agree
(NP)	Unsure	Unsure	Unsure	Agree	Unsure	Disagree
(N)	Agree	Agree	Agree	Agree	Agree	Agree
I recommend that magnesium therapy be used in patients during the post-MI period
(C)	No	No	No	No	No	No
(C)	No	No	No	No	No	No
(P)	No	Yes	Yes	Yes	Yes	Yes
(P)	No	Yes	Yes	Yes	Yes	Yes
(P)	No	No	No	No	No	No
(P)	No	Yes	Yes	Yes	Yes	Yes
(NP)	No	No	No	Yes	No	No
(N)	No	Yes	Yes	Yes	Yes	Yes

C: cardiologist, P: other physician, NP: non-practicing physician, N: non-physician
Each column contains the answers from different meta-analyses based on the number of randomized trials provided (top row). The total number of subjects in each of the meta-analyses is shown in the second row; the overall fixed effects odds ratio (OR) and random effects OR shown to the reviewers are given in rows 3 and 4 (the first trial only examined infarct size and there is no OR for mortality); and the I² value for heterogeneity is shown in row 5. There were three errors that were discovered after some reviewers had answered questions. The differences in the overall effect estimates were relatively minor and would not be expected to alter the responses by our reviewers. To remain transparent, we provide the numbers provided to the reviewers in this table, and the corrected numbers in Figure 1. The results for each question asked are shown in the subsequent rows. The choices for the first two questions were strongly disagree to strongly agree, and the choices for the third question were yes or no. In addition to the range of interpretations for any one meta-analysis, reviewer 3 moved from unsure to strongly disagree over the 6 meta-analyses for the second question whereas reviewer 7 moved in the opposite direction from strongly disagree to agree.

ISSN: 1472-6947