Skip to main content


Archived Comments for: Is the coverage of google scholar enough to be used alone for systematic reviews

Back to article

  1. Google Scholar as a single source for systematic reviews has not been justified

    Alison Weightman, Cardiff University

    13 February 2013

    This paper forms part of an important debate. Google Scholar (GS) is an increasingly powerful search tool that should rightly be considered by systematic review searchers. Research studies comparing GS to other established search tools are valuable.

    However we feel the paper fails to address some really important issues, and believe that the conclusion that GS could be a single search source for systematic reviews has not been justified.

    Being able to find a paper once you know about it is not the same as finding the paper in the first instance. Problems identified with this publication, on which we would welcome the authors' and journal editors' responses, are:

    1. Google Scholar is constantly updated and it isn't possible to know whether the references included in it at the search date for this paper would have been there at the search date of the candidate systematic reviews. GS trawls many sites and may have picked up these references as a result of their inclusion in the reviews.

    2. The lack of advanced search functions is acknowledged but there seems to be an implicit assumption in the paper that review authors would have chosen the right terms to pick up relevant publications, including citations without an abstract. Further, the very low precision rates measured by the authors of this paper from example searches (circa 0.1% precision) severely limit the current value of GS as a single search source for systematic reviews. It would ask a lot of systematic reviewers to trawl/sift 36,000 results in order to find 36 relevant papers.

    3. The authors did not require the title to link to an abstract or full text in Google Scholar (GS), to be regarded as a study within GS; a link to a citation was considered sufficient. Each title identified via a citation would need to be searched for in other database(s) to find an abstract, or obtained in full text to assess relevance. This would entail a substantial additional workload for systematic review authors which is not addressed in the paper.

    4. The paper has several sections that should have been edited for grammatically correct English. Errors are not surprising given that the authors are not writing in their native language, but it may call the editorial process into question.

    Alison Weightman, Fiona Morgan, Mala Mann and Bernadette Coles
    University Library Service, Cardiff University, UK

    Competing interests

    A competing interest exists when your professional judgment about a paper could possibly be influenced by considerations other than the paper's validity or importance. Detail possible competing interests here... No competing interests

  2. Interesting but...short shrift to expert searching

    Michelle Fiander, A personal comment

    13 February 2013

    That Google Scholar is a one stop source for searching because a selection of studies included in a selection of systematic reviews can be found in GS displays, I believe, faulty logic--perhaps an association fallacy. Whether or not I am correct regarding the logic, I am bit more concerned that the authors give very short shrift to the importance of the search interface (technology aspect) and knowledge of indexing (understanding gained by experience and training) which are both integral to searching for systematic reviews. While GS and other Web applications are absolutely fantastic (no argument from me on this point), neither the search interface, nor the scope and depth of GS is yet, in my opinion, sufficiently well defined or organized to support comprehensive searching for systematic reviews.

    Competing interests

    I am an Information Scientist, Trials Search Co-ordinator, for the Cochrane Collaboration Effective Practice and Organisation of Care Review Group and a member of the Cochrane Collaboration Steering Group.

  3. Of course GS will find what you already know exists

    Tuulevi Ovaska, University of Eastern Finland

    13 February 2013

    Dear colleagues,

    your article is very interesting but also rather confusing.

    It would naturally be very efficient and truly economic if only one simple and freely available search engine could be used to search and find all studies needed for a systematic review.

    You write that "GS was searched for all these studies one by one to assess the percentage of studies which could have been identified by searching only GS. ---All the 738 original studies included in the gold standard database were retrieved in GS (100%)."

    Well, of course, you were able to retrieve them all as you already knew what they were and searched them one by one.

    But would you have found them if you did not know them already? Would searching GS for the topic (instead of the studies already known) end in the result of the same studies? These are the important questions, imho.

    Competing interests

    None declared