Table 1 Collective intelligence study design
From: Collective intelligence in medical decision-making: a systematic scoping review
Study author, year | Description of experts | Real or simulated cases | Types of opinions aggregated | Study design | Relevance to collective intelligence |
|---|---|---|---|---|---|
Gagliardi, 2007 [25] | 20 general surgeons, 2 pathologists, 1 medical oncologist, 1 radiation oncologist | Real | Diagnosis, Treatment | Qualitative observational study to explore the role of multidisciplinary cancer conferences in practice | Describe collective output generated in multidisciplinary cancer conferences |
Douzgou, 2016 [24] | Physicians with patients with malformation syndromes | Real | Diagnosis | Descriptive study of a consultation tool which generates collective insight | Assess a collective intelligence tool |
Sternberg, 2017 [36] | International colleagues with urologic expertise | Real | Treatment | Use Twitter as a potential collective intelligence tool | Describe social media as a collective intelligence tool |
Sims, 2014 [35] | Clinicians affiliated with academic departments: 28 from pediatrics, 27 from neurology, 10 from internal medicine, 4 from psychiatric, 11 from pediatric neurology, 5 others | Real | Diagnosis, Treatment | Descriptive study of a clinical consultation system which generates collective insight and qualitative evaluation of the tool | Describe a collective intelligence tool |
Nault, 2009 [33] | 5 spinal deformity surgeons | Real | Treatment | Feasibility study of a surgical decision-making tool as compared to a group of experienced surgeons | Compare collective intelligence generated by experts with a technology tool |
Alby, 2015 [22] | 1 oncologist and others from hematology, anesthesiology, surgery, and nephrology | Real | Diagnosis | Qualitative observational study of conversations about cancer cases between the chief oncologist and other physicians at a hospital | Characterize collective intelligence generated in usual practice |
Kattan, 2013 [28] | 24 urologists and oncologists | Real | Prognosis | Analysis of physician group accuracy as compared to a nomogram | Compare collective intelligence generated by experts with a technology tool |
Kunina-Habenicht, 2015 [29] | 283 medical students, 20 expert physicians | Real | Diagnosis | Descriptive study of the development of a computerized test to assess diagnostic accuracy; results were compared among medical students and expert physicians | Compare computer-generated collective intelligence of experienced physicians to medical students |
Lajoie, 2012 [30] | 14 third-year medical students | Simulated | Diagnosis, Treatment | Qualitative observational study of team discussions with or without a technology tool to aid collaboration | Optimize metacognitive activities in collective intelligence with a technology tool |
Kalf, 1996 [27] | 21 geriatricians, 21 geriatric-psychiatrists, 21 internists | Simulated | Diagnosis | Analysis of diagnoses generated by different specialties | Compare collective intelligence among different specialists |
Larson, 1996 [31] | 24 first-year interns, 24 residents, 24 medical students | Simulated | Diagnosis | Qualitative observational study of team diagnostic discussions when teams are exposed to different case information | Characterize collective intelligence generated when groups have different amounts of information about a case |
Christensen, 2000 [23] | 24 first year interns, 24 residents, 24 medical students | Simulated | Diagnosis | Qualitative observational study of team diagnostic discussions when given different amounts of shared and unshared information | Characterize collective intelligence generated when groups have different amounts of information about a case |
Larson, 1998 [32] | 48 interns and 24 third-year medical students | Simulated | Diagnosis | Qualitative observational study of team diagnostic discussions when teams are exposed to different case information and given instructions about sharing information | Characterize collective intelligence generated when groups have different amounts of information about a case |
Semigran, 2016 [34] | 234 physicians, including fellows and residents | Simulated | Diagnosis | Analysis of a collective intelligence tool as compared to the accuracy of symptom checker websites | Compare a collective intelligence tool to online symptom checkers |
Hautz, 2015 [26] | 88 medical students | Simulated | Diagnosis | Analysis of diagnostic accuracy when participants worked in pairs or individually | Compare collective intelligence of pairs to individual aptitude |