- Research article
- Open Access
An analysis of online health information on schizophrenia or related conditions: a cross-sectional survey
BMC Medical Informatics and Decision Making volume 13, Article number: 98 (2013)
Around 20% of those who seek health information online, search specifically for mental health. However, little is known about the nature of the online health information offered by two European countries, Finland and Greece, which are characterized by markedly differing levels of Internet access and online health information seeking. This study aims to assess, describe and compare websites, written in two European, non-English languages (Finnish and Greek) that appear first after performing an online search concerning schizophrenia or related conditions.
The first 20 results from four search terms (searched in Finnish and Greek) in the Web search engine ‘Google’ were screened. A total of 160 websites were retrieved (80 Finnish, 80 Greek) and evaluated using a preformulated coding system which consisted of websites’ indicators, such as: types, characteristics, accountability, interactivity, aesthetics and content. Differences between websites were evaluated with Chi-Square or Fisher’s Exact tests for categorical data and independent t-tests for parametric data.
Twenty-four Finnish and thirty-four Greek websites (36% in total) were included. Almost two-thirds (62%, n=36) were owned by an organization, compared to 17% (n=10) by an individual. In both countries, aesthetics had the highest score (possible range 0–4, mean = 2.6, SD = .62), while interactivity the lowest (range 0–5, mean = 1.79, SD = .87). There were no statistically significant differences among the accountability, interactivity and aesthetics scores of the Finnish and Greek websites.
All assessed indicators suggest there is a need to improve Finnish and Greek online information about schizophrenia or related conditions. The poor website interactivity is of particular concern given the challenges faced by the target group. The findings can be used to guide the development and dissemination of online mental health information aimed at Finnish and Greek online health-seekers.
Schizophrenia is a chronic, severe, debilitating mental disorder [1, 2], which ranks third in global burden among the mental disorders . Approximately 25 million people are diagnosed with schizophrenia globally , about 2.5 million in the USA  and around 3.7 million in Europe . Schizophrenia affects the lives of those with the disorder and their caregivers [7, 8] due to hallucinations, perceptual and communication difficulties . The illness itself is also connected with high levels of fear, shame, guilt , and social isolation , which may lead to an unwillingness to adhere to treatment , consequent relapse  and readmissions [8, 13].
A Cochrane review  concluded that psychoeducation may promote adherence to medication, diminish relapses, reduce the length of hospitalization and rates of readmission for those with schizophrenia. Advances in technology allow psychoeducation to be delivered conveniently through the Internet . Such technology is now frequently used by members of the public to access health information online  with a substantial percentage of all online health searches being for mental health information . Internet users visit medical websites primarily to acquire information about a health condition, its treatment, symptoms, and to obtain health advice .
Recent evidence suggests that the Internet is also an important source of illness-related information for people with schizophrenia  who value the availability of reliable, evidence-base information on the Internet  and the role it can play in increasing their sense of empowerment by increasing their understanding of their illness and capacity to communicate with their doctors . In addition, web-based psychoeducation for people with schizophrenia shows some promise as a means of improving mental state, providing social support, and supporting medication compliance .
Schizophrenia is commonly associated with deficits in attention, concentration, visual perception and interpretation [1, 9]. At the same time, online health information can be easily misinterpreted  and online health seekers may be exposed to contradictory medical advice and opinions . As a consequence, it is particularly important that websites providing information about schizophrenia should present simple but high quality and understandable information, in a design format that takes into account the above difficulties [23, 24].
The Internet is a potentially useful tool for retrieving information about mental illness [25, 26]. In the USA, it is estimated that 21% of information health-seekers use the Internet for mental health issues . In the UK, it has been reported that about 18% of all Internet users and 31.5% of Internet users with a past history of mental health problems used the Internet for retrieving mental health information . However, patterns of computer and Internet use vary substantially across countries . Two-thirds of the total population in the European Union (EU) use the Internet at least once a week , but there are marked discrepancies in Internet usage between member states of the EU. For example, in Southern Europe and specifically in Greece, 50% of households have Internet access, while the percentage in a Northern European country, such as Finland, is 84% . Further, in 2011, 58% of Finnish citizens used the Internet to search for health-related information (second after Icelanders), whereas Greek citizens (30%) are among the least likely of the European Union members (after Italy, Cyprus and Romania) to use the Internet for health information seeking . While a basic cornerstone of the European agenda is the right of freedom of movement across the EU , support of homogeneous rights, quality health services for citizens , and intercountry collaboration , priority should be given ensuring the availability and universal accessibility of high quality online mental health information [34–36]. The quality of the provided online health or mental health information, worldwide and specifically in Europe [37, 38] still remains unclear. To our knowledge, four studies of the quality of online health information have been conducted in Finland [39, 40], and Greece [41, 42]. Only one of these was related to mental health. It found that Finnish websites providing information about antidepressants did not cover all aspects of treatment . These two European countries differ in population, Internet access and, Information and Communication Technologies (ICT) use and attitudes [28–30]. The objective of the current study was to assess, describe and compare Finnish and Greek websites that first appear after performing an online search concerning schizophrenia or related condition.
Data collection and sample
On November 30, 2011, the Web search engine ‘Google’ was used to identify online health information on schizophrenia or related conditions. Google was selected because it is the most frequently used search engine  and the one of the most likely to be used by someone searching for online health information. Country specific versions for Finland (http://www.google.fi/) and Greece (http://www.google.gr/) were employed.
Search terms in Finnish and Greek language were chosen with the aim of generating a list of websites that might be similar to a search produced by a Finnish or Greek adult with average medical, Internet, and computer expertise [34, 44, 45]. Since searches are often triggered by a particular diagnosis or condition [17, 46], we hypothesized that the four most probable search terms for someone within the spectrum of schizophrenic disorders would be: mental illness (‘mielisairaus’/‘ψυχική ασθένϵια’), mental disorder (’mielenterveyden häiriö’/‘ψυχική διαταραχή’), schizophrenia (‘skitsofrenia’, ‘σχιζφφρένϵια’), and psychosis (‘psykoosi’/‘ψύχωση’) respectively in Finnish and in Greek language.
The first 20 websites returned by the search engine for each of the four search terms were examined for eligibility (N = 160; 80 Finnish and 80 Greek). It was considered unlikely that the typical consumer would click on more than 20 results [47–49] from a single search. The 20 direct links from each ‘Google’ search term were saved. Screenshots of the direct webpages appearing from every ‘Google’ result were taken.
Additionally, within every website five webpages were collected, applying convenience sampling. After opening and saving the first webpage that appeared from Google results, then four additional randomly selected webpages were saved which were linked to the initial page. Convenience sampling was preferred for the selection of webpages  because of their convenient accessibility and proximity.
To be included in the study, a valid website was required to satisfy the following criteria: 1) focused on health or mental health issues for adults in the Finnish or Greek language; and 2) was a standard information website, open web-based encyclopaedia (e.g. ‘wikipedia’), discussion forum, blog or wiki. The latter was included since it has been reported that they are favoured online sources among people with schizophrenia . Websites were excluded if they: 1) were not focused on health or mental health issues; 2) targeted educators or special schools’ educators or described courses; 3) primarily involved advertisements or book promotion; 4) were links leading to external files (e.g. .doc, .pdf, .ppt); 5) were invalid addresses or malware; 6) were incidental mental health articles or discussion in a non-health oriented forum or blog; 7) were not written in the Finnish or Greek languages; 8) or provided health information for a non-adult population (e.g. for children or adolescents, or their parents).
A total of 56 Finnish websites and 46 Greek websites failed to satisfy the inclusion criteria and were excluded (Figure 1). This left 24 Finnish and 34 Greek language websites (58 sites in total) for analysis.
Coding system and instrument
The coding system used for the data categorization consisted of six indicators: 1) type of website; 2) characteristics; 3) accountability; 4) interactivity; 5) aesthetics; and 6) content.
First, websites were categorized in seven types according to an adaptation of Morel et al.  (Table 1). Second, characteristics of the sites were evaluated according to criteria used by Griffiths & Christensen [34, 45]. Three additional characteristics were added including other languages available, provision of online services (e.g. video-conferencing), and presence of the Health On the Net (HON) code  which is an ethical standard certification of the trustworthiness of a specific health and medical website (Table 1). Third, accountability was evaluated with the adapted version of Silberg et al’s  scale  (Table 1). The maximum total score of this scale is 9 (possible range 0–9). Fourth, interactivity was evaluated with the adapted version of Khazaal et al. [54, 55] (Table 1). The maximum score on this scale is 5 (range 0–5). Fifth, aesthetics was evaluated with the adapted version of Kisely et al.’s criteria  originally introduced by Abbott  (Table 1). The maximum score on this scale is 4 (range 0–4).
Lastly, the content of the provided online information was evaluated based on the availability of answers to health-related enquiries frequently generated by people with psychiatric disorders, such as information about diagnosis, treatment options, and medication side-effects . Answers were searched using the screenshots of the direct webpages and not through website navigation. This decision was made because online health-seekers do not spend a lot of time searching for information within a website [47, 57], and people with schizophrenia face difficulties with website navigation [23, 24]. The searched items were: 1) diagnosis; 2) treatment; 3) information about patient associations; and 4) information about clinics (Table 1). If a webpage did not specifically state ‘symptom’ or ‘diagnosis’ of schizophrenia or related mental disorders, the page was considered not to provide information about symptoms or diagnosis, although these may have incorporated an obscure reference to this information. In addition, if the term ‘treatment’ or ‘diagnosis’ was mentioned (e.g. ‘The quicker the psychosis is diagnosed and the treatment is started, the better the recovery’) but a specific treatment option was not mentioned (e.g. ‘The treatment consists of medication and psychotherapy’), then it was not considered to contain treatment information. This decision was taken because people with schizophrenia may require concrete descriptions  to understand the text they are reading.
Coding was conducted as follows. First, researchers independently familiarized themselves with the retrieved websites. Second, they undertook an evaluation of website type, characteristics, accountability, interactivity, aesthetics, (CA for Greek and SS for Finnish data) and content of the webpages (CA and CB for Greek and MV and HH for Finnish data), with each website score on each of the criteria with ‘1’ indicating the criterion was satisfied and ‘0’ that it was absent. In addition, type, characteristics, disclosure, currency, interactivity were determined through navigation within the website, while authorship, attribution, and aesthetics were scored according to 5 random webpages within a single website. This approach was adopted, since in many cases, a single website was diverse (e.g. name of author was mentioned on one page but not the author of a text on another page). Content was evaluated only through the direct webpage retrieved from the ‘Google’ search, with the aim of evaluating that part of the website most health-seekers were likely to access (Table 1).
Reliability of the coding was ensured by following actions. First, two raters (CA, SS) critically examined the Greek and Finnish ratings concerning website type, characteristics, accountability, interactivity and aesthetics. The evaluators discussed any discrepancies in the decisions made with the final rating made by agreement. Reliability was investigated by recoding (third independent rater) a random selection of 20% of the websites. The number of scoring errors (n = 65) was then divided by the number of coded cells (N = 2538), yielding an error rate of 2.56%. Second, inter-rater reliability was assessed separately for the Finnish and Greek analysis. The percent agreement was calculated by dividing the number of observations in which the raters agreed by the total observations (97%). Third, any scoring disagreements in content analysis were resolved through discussion between the two same-language raters. Percent agreement for content analysis was 80% and 83% for the Finnish and Greek data respectively .
Accountability, interactivity, and aesthetics indicator scores were computed by summing the value of each score of the item (yes = 1; no = 0). Correlations among accountability, interactivity and aesthetics were analyzed using Pearson’s correlation coefficient. Differences between categorical variables was evaluated using Chi-Square tests or Fisher’s Exact test if expected cell counts were <5. Differences in indicator scores between Finnish and Greek websites were analysed using independent t-tests. SPSS version 19.0 (IBM Corporation, Somers, NY, USA) was used for all statistical analyses and in all tests, an alpha of .05 was employed for identifying a statistically significant difference.
The 58 included websites were: commercial (n = 18), personal pages developed by health professionals (n = 7), university websites (n = 1), non-profit organizations (n = 7), governmental (n = 2), open source (n = 5) websites where anyone could modify the webpage content, for example ‘wikis’ or online encyclopedias, or other sources (n = 18), for example, blogs or patient/professional associations.
Characteristics of the websites
Fifty-five of the websites originated from Finland or Greece (24 Finnish, 31 Greek) and three were maintained by the European Commission. About one third (34%, n = 11) of Finnish and Greek (29%, n = 7) websites provided other language options, typically, English and/or Swedish in Finland and, English and/or French in Greece. Online services, e.g. video-conferencing or web telephony counseling were provided by 6% of Greek websites (n = 2), but not by Finnish websites. Three Greek websites but none of the Finnish websites had the HON certification.
Comparing Finnish and Greek website characteristics, there were statistically significant differences for three factors. First, a greater percentage of the Greek than Finnish websites were owned by a health professional (56% vs. 21%, p = .008). Second, there was a higher probability that a Greek website involved a health professional in the content development (76% vs. 38%, p = .003). Third, Greek websites promoted products or services more often than their Finnish counterparts (50% vs. 21%, p = .024) (Table 2).
Accountability of the websites
Most of the websites specified the site ownership (74%, n = 43) and over half (55%, n = 32) provided sources for the content. Eleven websites mentioned the authors’ affiliations and the last date of modification. In general, accountability of the websites was poor: the mean value of the accountability score was 3.33 (SD 1.93) out of a possible maximum 9 (Table 3).
When accountability of Finnish and Greek websites was compared, only one statistically significant difference was found: authorship was specified more often in Greek than Finnish websites (47% vs. 21%, p = .04) (Table 3).
Interactivity of the websites
Five websites (9%) incorporated evaluation questionnaires, for example, to enable the user to provide feedback about the website or to evaluate his/her health status. About three-quarters (74%, n = 43) of all websites provided the e-mail address of the webmaster and two-third (62%, n = 36) provided an intra-site search engine. The total mean interactivity score was 1.79 (SD .87, maximum 5) which was low with no statistically significant differences between the countries (Table 4).
Aesthetics of the websites
All the websites used headings or subheadings (n = 58). Almost half of the sites did not incorporate advertisements. Two-thirds of all sites had hyperlinks to external sites, while four sites (7%) included diagrams in their content. The mean score for the total aesthetics indicator was 2.6 (SD .62, maximum 4). There were no statistically significant differences in the aesthetic ratings for the Finnish and Greek websites (Table 5).
Content of the direct webpages
With respect to the availability of answers to five popular enquiries, the most commonly answered in Finnish direct webpages concerned diagnosis (71%), while in Greek direct webpages information about treatment (32%) was the most common. The only statistically significant difference was in the provided information about diagnosis (p<.001) (Table 6).
Overall, proportionate to the potential range of scoring, the highest scoring indicator was aesthetics of the websites (2.6 out of 4), followed by accountability (3.33 out of 9) and interactivity (1.79 out of 5).
This study was designed to assess, describe and compare Finnish and Greek websites appearing first on ‘Google’ when using a search term on schizophrenia or related conditions. The websites were analysed with respect to type and characteristics, accountability, interactivity, aesthetics and content of the websites. It seems that first-appearing Finnish and Greek websites, providing mental health-related information were of low quality (reflecting the fact that few of the Silberg criteria, on average, were satisfied) with respect to the assessed indicators. These results are similar to those reported in Reavley and Jorm’s review  of 23 studies of the quality of mental disorder information websites. The current findings are valuable because there is evidence that people with schizophrenia perceive the Internet as an important and influential source of information . Although research interest in the quality of online health information has been apparent for at least 17 years [59, 60], little seems to have changed regarding the development and dissemination of online health information.
Although compared to Greek citizens, Finnish citizens are much more likely to seek health information online , and to have greater access to the internet , and more advanced ICT skills [28, 30], our study found no evidence that the first-returned Finnish websites providing schizophrenia-related information were of better quality than Greek websites. This finding does not exclude the possibility that there are high quality Finnish and Greek mental health websites, but it does demonstrate that if high quality sites exist they are not among the first 20 results when a schizophrenia-related Google search is performed.
In the present analysis, accountability of Greek websites was higher than the Finnish ones. This could be due to the profusion of private doctors and private diagnostic clinics in Greece , who use websites to advertise their professional profile and attract consumers. This is supported by the finding that service promotion was more common on the Greek websites than on Finnish websites. In contrast to their Finnish counterparts, most Greek consumers tend to pay privately for their healthcare . The latter may also explain the significantly greater level of health professional ownership and professional content development of mental-health related websites in Greece compared to Finland. Identification of the ownership structure of the Finnish websites was unclear. There were no significant differences between the Finnish and Greek websites with respect to other characteristics of websites such as scope, country and whether a drug company was involved.
Website interactivity was poor in both countries. Interactivity is considered an important website element offering a way to communicate with users and engage them in using the provided services . It is unclear if the present findings reflect the poor interactivity of websites, in general, or if they reflect a more specific limitation of mental health websites in a field where mental health consumers are often inaccurately perceived as passive users of health services . Providing consumers with trustworthy online health information has the potential to lead to better outcomes when given information and choices [65, 66]. Our findings also suggest that the Finnish and Greek websites are primarily designed for passive information delivery, rather than providing interactive tools and support. It is critical to consider consumers’ needs and opinions , if the Internet is to be an effective psychoeducation tool . Moreover, supporting the inclusion of vulnerable populations, like people with schizophrenia, into an information society is emphasized by the European Commission .
Our analysis demonstrated that the content of the webpages deriving directly from the first 20 ‘Google’ results, and in particular whether they provided answers to common consumer enquiries, was low. It has been reported [25, 54, 68] that online health-seekers search for information related to a disease, symptoms, treatment options, clinics and patients’ associations. In the current study it was found that this information was typically not available within the first webpage of the included websites.
In our study we included ‘wikis’ in the analysis, although their use as a reliable health resource has been criticized [69, 70]. This decision was made on the grounds that web search engines often lead users to Wikipedia . Wikis are favored online sources among consumers with schizophrenia . Large amounts of health information are provided in Wikipedia , and there has been some evidence that health professionals believe that popular Wikipedia articles (including those on ‘schizophrenia’) are of ‘good quality’ . Such evaluations are typically focused on the accuracy of the content rather than on the interactivity or aesthetics of the site. While accurate content is necessary , it is also crucial to ensure that information is delivered in a way that a person with schizophrenia or a related condition can become informed, since symptoms of these disorders, such as attention deficit or delusional misinterpretations, may compromise Internet use . We attempted to assess all generated websites such as wikis, blogs, forums, etc., regardless of their structure, in order to focus on those websites that are returned in typical online searches for mental health information. However, future research should consider investigating comparative quality across these different types of resources including wikis.
There are a number of limitations of this study. First, the online search was performed in November 2011 and considering the changing nature of the Internet, it is unsure that the results of this study will remain the same in the future. Second, there are no studies on how Finnish and Greek people diagnosed with schizophrenia retrieve online mental health information. Hence, they may use a search engine other than ‘Google’ or use different search terms or methods to acquire online health information about their illness. In addition, the conveniece sampling method used for the selection of five webpages for each website, does not ensure that someone could access and read the specific five webpages. Third, the information gathered about Finnish and Greek websites used a limited range of specific quality measures, excluding, for example, readability  and quality of content [45, 56, 72]. In Finland, official clinical guidelines for mental disorders are available, but not in Greece. For this reason, the current study did not employ a measure of the quality of content based on evidence-based guidelines. Fourth, the quality indicators of this study could potentially consist of more items. For example, the aesthetics indicator did not measure the presence or not of images or video or other media (only diagrams) in our included webpages, although such material has the potential to assist people to better understand the text they read . Fifth, the availability of answers in common inquiries (diagnosis, treatment, associations, clinics) was assessed from the direct webpages. Therefore, the content description is limited to one webpage only and, thus, does not apply to the content of the whole website. Additionally, only the presence or absence of specific content was assessed (diagnosis, treatment, patient associations, clinics), without evaluating the accuracy or comprehensiveness of the provided information. Thus, even when the webpage achieved a positive rating on the current scoring system, the content of the webpage might have been of low evidence-based quality. Last, inter-rater reliability was only calculated for the content of webpages, since it was the only indicator assessed by two same-language raters. However, the coding of the other indicators was discussed for all data, between the Finnish and the Greek rater.
All assessed indicators of Finnish and Greek websites, appearing first on ‘Google’ when using a schizophrenia-related search term achieved low scores. This is despite the difference in ICT context in the two countries and in particular the greater Internet use and access, and ICT experience in Finland. Regardless of the latter disparities between Finland and Greece, those with schizophrenia and related conditions receive a similar level of online experience when they spontaneously search for schizophrenia-related information via ‘Google’.
This study provides a foundation for the future development of websites on the topic of schizophrenia, and suggests that improvement in many aspects of website quality is needed. Although the quality of mental health information websites may have improved over the past decade , the current findings suggest the need for increased awareness about the various quality indicators among website developers, bloggers, ‘wiki’, forum contributors, and others who upload schizophrenia-related content online.
All developers, including health professionals who are significant online health information contributors  should be encouraged to comply with international standards and guidelines before uploading health-related information. Such instruments provide an essential tool to guide developers to produce usable websites including those intended for people with schizophrenia. This could promote homogeneity, easy access, and clear interpretation of the online health information, and thereby, support and educate consumers and their families. Additionally, health professionals should be open to provide guidance on which online health information can be trusted and why, since people diagnosed with a mental disorder are also searching for health information online .
Furthermore, the most popular search engines could facilitate the delivery of high quality websites. For example, a search engine might potentially be programmed to return high quality information websites [35, 74] prior to sponsored results and lesser quality websites. Moreover, Google’s country specific versions might follow Google USA’s example , where they partnered with the National Institute of Health to generate relevant health-related information in response to consumers’ searches .
Information and communication technologies.
Freedman R: Schizophrenia. N Engl J Med. 2003, 349: 1738-1749. 10.1056/NEJMra035458.
Chan SW, Hsiung PC, Thompson DR, Chen SC, Hwu HG: Health-related quality of life of Chinese people with schizophrenia in Hong Kong and Taipei: a cross-sectional analysis. Res Nurs Health. 2007, 30: 261-269. 10.1002/nur.20189.
Collins PY, Patel V, Joestl SS, March D, Insel TR, Daar AS: Grand challenges in global mental health. Nature. 2011, 475: 27-30. 10.1038/475027a.
Bloom DE, Cafiero ET, Jané-Llopis E, Abrahams-Gessel S, Bloom LR, Fathima S, Feigl AB, Gaziano T, Mowafi M, Pandya A, Prettner K, Rosenberg L, Seligman B, Stein AZ, Weinstein C: The Global Economic Burden of Noncommunicable Diseases. 2011, Geneva: World Economic Forum,http://www3.weforum.org/docs/WEF_Harvard_HE_GlobalEconomicBurdenNonCommunicableDiseases_2011.pdf,
Mental Health America: Schizophrenia: What you need to know. 2012,http://www.nmha.org/go/information/mental-health-info/schizophrenia/schizophrenia-what-you-need-to-know/schizophrenia-what-you-need-to-know,
The European Multi-Centre Association Study of Schizophrenia (EMASS): Bad luck and -the legacy of schizophrenia. [http://ec.europa.eu/research/success/en/med/0285e.html]
Palazzolo J, Brousse G, Favre P, Llorca PM: The information of the schizophrenic patient: actuality. Encéphale. 2005, 31: 227-234.
Spaniel F, Vohlídka P, Hrdlicka J, Kozený J, Novák T, Motlová L, Cermák J, Bednarík J, Novák D, Höschl C: ITAREPS: Information Technology Aided Relapse Prevention Programme in Schizophrenia. Schizophr Res. 2008, 98: 312-317. 10.1016/j.schres.2007.09.005.
Wible CG, Preus AP, Hashimoto R: A cognitive neuroscience view of schizophrenic symptoms: abnormal activation of a system for social perception and communication. Brain Imaging Behav. 2009, 3: 85-110. 10.1007/s11682-008-9052-1.
Miller R, Mason SE: Shame and guilt in first-episode schizophrenia and schizoaffective disorders. J Contemp Psychother. 2005, 35: 211-221. 10.1007/s10879-005-2701-4.
Goff DC, Hill M, Freudenreich O: Strategies for improving treatment adherence in schizophrenia and schizoaffective disorder. J Clin Psychiatry. 2010, 71 (Suppl 2): 20-26. 10.4088/JCP.9096su1cc.04.
Geddes J: Prevention of relapses in schizophrenia. N Engl J Med. 2002, 346: 56-58. 10.1056/NEJM200201033460112.
McIntosh A, Conlon L, Lawrie S, Stanfield AC: Compliance therapy for schizophrenia. Cochrane Database Syst Rev. 2006, 3: CD003442
Xia J, Merinder LB, Belgamwar MR: Psychoeducation for schizophrenia. Cochrane Database Syst Rev. 2011, 6: CD002831
Rotondi AJ, Anderson CM, Haas GL, Eack SM, Spring MB, Ganguli R, Newhill C, Rosenstock J: Web-based psychoeducational intervention for persons with schizophrenia and their supporters: one-year outcomes. Psychiatr Serv. 2010, 61: 1099-1105. 10.1176/appi.ps.61.11.1099.
Fox S, Duggan M: Health Online 2013. 2013, Pew Internet & American Life Project,http://pewinternet.org/Reports/2013/Health-online.aspx,
Fox S, Jones S: The Social Life of Health Information. 2009, Pew Internet & American Life Project,http://www.pewinternet.org/~/media//Files/Reports/2009/PIP_Health_2009.pdf,
Shuyler KS, Knight KM: What are patients seeking when they turn to the Internet? Qualitative content analysis of questions asked by visitors to an orthopaedics Web site. J Med Internet Res. 2003, 5: e24-10.2196/jmir.5.4.e24.
Schrank B, Sibitz I, Unger A, Amering M: How patients with schizophrenia use the internet: qualitative study. J Med Internet Res. 2010, 12: e70-10.2196/jmir.1550.
Välimäki M, Hätönen H, Lahti M, Kuosmanen L, Adams CE: Information and communication technology in patient education and support for people with schizophrenia. Cochrane Database Syst Rev. 2012, 10: CD007198
Cline RJ, Haynes KM: Consumer health information seeking on the Internet: the state of the art. Health Educ Res. 2001, 16: 671-692. 10.1093/her/16.6.671.
Hartzband P, Groopman J: Untangling the Web - patients, doctors, and the Internet. N Engl J Med. 2010, 362: 1063-1066. 10.1056/NEJMp0911938.
van der Krieke L, Emerencia AC, Aiello M, Sytema S: Usability evaluation of a web-based support system for people with a schizophrenia diagnosis. J Med Internet Res. 2012, 14: e24-10.2196/jmir.1921.
Valimaki M, Anttila M, Hatonen H, Koivunen M, Jakobsson T, Pitkanen A, Herrala J, Kuosmanen L: Design and development process of patient-centered computer-based support system for patients with schizophrenia spectrum psychosis. Inform Health Soc Care. 2008, 33: 113-123. 10.1080/17538150802127207.
Fox S: Online Health Search 2006. 2006, Pew Internet & American Life Project,http://www.pewinternet.org/~/media//Files/Reports/2006/PIP_Online_Health_2006.pdf.pdf,
Kalk NJ, Pothier DD: Patient information on schizophrenia on the internet. Psychiatric Bulletin. 2008, 32: 409-411. 10.1192/pb.bp.107.017004.
Powell J, Clarke A: Internet information-seeking in mental health: population survey. Br J Psychiatry. 2006, 189: 273-277. 10.1192/bjp.bp.105.017319.
Eurostat: Use of ICTs and use of on-line services, 2008–2010 (% of individuals aged 16 to 74). 2010,http://epp.eurostat.ec.europa.eu/statistics_explained/index.php?title=File:Use_of_ICTs_and_use_of_on-line_services,_2008-2010_(%25_of_individuals_aged_16_to_74).png&filetimestamp=20111117095543,
Seybert H: Internet use in households and by individuals in 2011. 2011,http://epp.eurostat.ec.europa.eu/cache/ITY_OFFPUB/KS-SF-11-066/EN/KS-SF-11-066-EN.PDF,
Eurostat: Individuals using the Internet for seeking health-related information - Percentage of individuals aged 16 to 74. 2011,http://epp.eurostat.ec.europa.eu/tgm/table.do?tab=table&plugin=0&language=en&pcode=tin00101,
Eurofound - European Foundation for the Improvement of Living and Working Conditions: Mobility in Europe – The way forward. 2007, European Foundation for the Improvement of Living and Working Conditions,http://www.eurofound.europa.eu/pubdocs/2007/03/en/1/ef0703en.pdf,
Reding V: Commission recommendation on cross-border interoperability of electronic health record systems. 2008,http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2008:190:0037:0043:EN:PDF,
European Commission, Information society and media directorate – General: Report on the public consultation on eHealth Action Plan 2012–2020. 2012,http://www.ehealthnews.eu/images/stories/pdf/ehap2012public-consult-report.pdf,
Griffiths KM, Christensen H: The quality and accessibility of Australian depression sites on the World Wide Web. Med J Aust. 2002, 176: S97-S104.
Theodosiou L, Green J: Emerging challenges in using health information from the internet. Adv Psychiatr Treat. 2003, 9: 387-396. 10.1192/apt.9.5.387.
America’s Health Literacy: Why We Need Accessible Health Information. An Issue Brief From the U.S. Department of Health and Human Services. 2008,http://www.health.gov/communication/literacy/issuebrief/,
Eysenbach G, Powell J, Kuss O, Sa ER: Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA. 2002, 287: 2691-2700. 10.1001/jama.287.20.2691.
Reavley NJ, Jorm AF: The quality of mental disorder information websites: a review. Patient Educ Couns. 2011, 85: e16-e25. 10.1016/j.pec.2010.10.015.
Prusti M, Lehtineva S, Pohjanoksa-Mäntylä M, Bell JS: The quality of online antidepressant drug information: an evaluation of English and Finnish language Web sites. Res Social Adm Pharm. 2012, 8: 263-268. 10.1016/j.sapharm.2011.03.003.
Tallgren M, Bäcklund M: Patient information about general anaesthesia on the internet. Anaesthesia. 2009, 64: 408-415. 10.1111/j.1365-2044.2008.05813.x.
Bamidis P, Kerassidis F, Pappas K, Health information on the internet : evaluating greek health portals and depicting users' attitudes in west macedonia, Greece. Stud Health Technol Inform. 2005, 116: 885-890.
Patsioura F, Kitsiou S, Markos A: Evaluation of Greek public hospital websites. In Proceedings of the International Conference on E-business: 7-10 July 2009; Milan. 2009, 223-229. INSTICC Press;
Purcell K, Brenner J, Rainie L: Search Engine Use 2012. 2012,http://pewinternet.org/~/media//Files/Reports/2012/PIP_Search_Engine_Use_2012.pdf,
Khazaal Y, Fernandez S, Cochand S, Reboh I, Zullino D: Quality of web-based information on social phobia: a cross-sectional study. Depress Anxiety. 2008, 25: 461-465. 10.1002/da.20381.
Griffiths KM, Christensen H: Quality of web based information on treatment of depression: cross sectional survey. BMJ. 2000, 321: 1511-1515. 10.1136/bmj.321.7275.1511.
Morahan-Martin JM: How internet users find, evaluate, and use online health information: a cross-cultural review. Cyberpsychol Behav. 2004, 7: 497-510.
Eysenbach G, Köhler C: How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002, 324: 573-577.
Eysenbach G, Köhler C: Does the internet harm health? Database of adverse events related to the internet has been set up. BMJ. 2002, 324: 239-10.1136/bmj.324.7331.239.
Höchstötter N, Lewandowski D: What users see – structures in search engine results pages. Inform Sciences. 2009, 179: 1796-1812. 10.1016/j.ins.2009.01.028.
Fricker RD: Sampling methods for web and e-mail surveys. The SAGE handbook of Online Research Methods. Edited by: Fielding NG, Lee RM, Blank G. 2008, London: SAGE, 195-216.
Morel V, Chatton A, Cochand S, Zullino D, Khazaal Y: Quality of the web-based information on bipolar disorder. J Affect Disord. 2008, 110: 265-269. 10.1016/j.jad.2008.01.007.
Health On the Net foundation (HON): The commitment to reliable health and medical information on the internet. 2011,http://www.hon.ch/HONcode/Patients/Visitor/visitor.html,
Silberg WM, Lundberg GD, Mussachio RA: Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor–Let the reader and viewer beware. JAMA. 1997, 277: 1244-1245. 10.1001/jama.1997.03540390074039.
Khazaal Y, Chatton A, Cochand S, Hoch A, Khankarli MB, Khan R, Zullino DF: Internet use by patients with psychiatric disorders in search for general and medical informations. Psychiatr Q. 2008, 79: 301-309. 10.1007/s11126-008-9083-1.
Abbott VP: Web page quality: can we measure it and what do we find? A report of exploratory findings. J Public Health Med. 2000, 22: 191-197. 10.1093/pubmed/22.2.191.
Kisely S, Ong G, Takyar A: A survey of the quality of web based information on the treatment of schizophrenia and Attention Deficit Hyperactivity Disorder. Aust N Z J Psychiatry. 2003, 37: 85-91. 10.1046/j.1440-1614.2003.01107.x.
Crespo J: Training the health information seeker: quality issues in health information web sites. Libr Trends. 2004, 53: 360-374.
Lombard M, Snyder-Duch J, Bracken CC: Content analysis in mass communication: assessment and reporting of intercoder reliability. Hum Commun Res. 2002, 28: 587-604. 10.1111/j.1468-2958.2002.tb00826.x.
Bower H: Internet sees growth of unverified health claims. BMJ. 1996, 313: 381-
Pealer LN, Dorman SM: Evaluating health-related Web sites. J Sch Health. 1997, 67: 232-235. 10.1111/j.1746-1561.1997.tb06311.x.
The Linked World: Reasearch Report. How ICT Is Transforming Societies, Cultures, and Economies. 2008,http://pressoffice.telefonica.com/documentos/LibroProductividad_VersionIngles.pdf,
Tountas Y, Karnaki P, Pavi E, Souliotis K: The “unexpected” growth of the private health sector in Greece. Health Policy. 2005, 74: 167-180. 10.1016/j.healthpol.2005.01.013.
Thompson S, Massialos E: What are the equity, efficiency, cost containment and choice implications of private health-care funding in western Europe?. 2004, WHO Regional Office for Europe’s Health Evidence Network (HEN),http://www.euro.who.int/__data/assets/pdf_file/0003/74694/E83334.pdf,
Koo C, Wati Y, Park K, Lim MK: Website quality, expectation, confirmation, and end user satisfaction: the knowledge-intensive website of the Korean National Cancer Information Center. J Med Internet Res. 2011, 13: e81-10.2196/jmir.1574.
Adams JR, Drake RE, Wolford GL: Shared decision-making preferences with severe mental illness. Psychiatr Serv. 2007, 58: 1219-1221. 10.1176/appi.ps.58.9.1219.
Griffiths KM, Christensen H, Jorm AF, Evans K, Groves C: Effect of web-based depression literacy and cognitive–behavioural therapy interventions on stigmatising attitudes to depression. Br J Psychiatry. 2004, 185: 342-349. 10.1192/bjp.185.4.342.
COM: Together for Health: A Strategic Approach for the EU 2008–2013. 2007,http://ec.europa.eu/health/ph_overview/Documents/strategy_wp_en.pdf,
Kortum P, Edwards C, Richards-Kortum R: The impact of inaccurate Internet health information in a secondary school learning environment. J Med Internet Res. 2008, 10: e17-10.2196/jmir.986.
Czarnecka-Kujawa K, Abdalian R, Grover SC: The quality of open access and open source Internet material in gastroenterology: is Wikipedia appropriate for knowledge transfer to patients?. Gastroenterology. 2008, 134: A325-A326.
Heilman JM, Kemmann E, Bonert M, Chatterjee A, Ragar B, Beards GM, Iberri DJ, Harvey M, Thomas B, Stomp W, Martone MF, Lodge DJ, Vondracek A, de Wolff JF, Liber C, Grover SC, Vickers TJ, Meskó B, Laurent MR: Wikipedia: a key tool for global public health promotion. J Med Internet Res. 2011, 13: e14-10.2196/jmir.1589.
Laurent MR, Vickers TJ: Seeking health information online: does Wikipedia matter?. J Am Med Inform Assoc. 2009, 16: 471-479. 10.1197/jamia.M3059.
Khazaal Y, Chatton A, Cochand S, Coquard O, Fernandez S, Khan R, Billieux J, Zullino D: Brief DISCERN, six questions for the evaluation of evidence-based content of health-related websites. Patient Educ Couns. 2009, 77: 33-37. 10.1016/j.pec.2009.02.016.
Johnson-Glenberg MC: Web-based training of metacognitive strategies for text comprehension: focus on poor comprehenders. Read Writ. 2005, 18: 755-786. 10.1007/s11145-005-0956-5.
Griffiths KM, Tang TT, Hawking D, Christensen H: Automated assessment of the quality of depression websites. J Med Internet Res. 2005, 7: e59-10.2196/jmir.7.5.e59.
Feufel MA, Stahl SF: What do web-use skill differences imply for online health information searches?. J Med Internet Res. 2012, 14: e87-10.2196/jmir.2051.
Andrews J: Google partners with NIH: Government gets top spot. 2010,http://www.contactdd.com/DDDocuments/August_2010.pdf,
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/13/98/prepub
We gratefully acknowledge the help provided by Christina Boundouvis with data analysis of the Greek webpages. This research was supported by the Academy of Finland (132581), the Finnish Cultural Foundation, and the Operational Program Education and Lifelong Learning of the National Strategic Reference Framework (NSRF) 2007–2013, within the framework of the Program for personalized evaluation process - Horizontal Act project, funded by the European Union (European Social Fund-ESF) and the Public Investment Program (PIP), and implemented by the State Scholarship Foundation (2011‐2‐162; IKY, Greece). The funding sources had no involvement in the study.
The authors declare that they have no competing interests.
CA led the conception and design, acquisition of data, analysis and interpretation of data, drafted and revised the manuscript and gave final approval of the version to be published. HH made substantial contributions to analysis and interpretation of data, was involved in drafting the manuscript or revising it critically for important intellectual content and gave final approval of the version to be published. SS made substantial contributions to acquisition and analysis of data and gave final approval of the version to be published. CL made substantial contributions to conception, interpretation of data, was involved in drafting the manuscript or revising it critically for important intellectual content and gave final approval of the version to be published. KG made substantial contributions to conception, interpretation of data, was involved in drafting the manuscript or revising it critically for important intellectual content, and gave final approval of the version to be published. MV made substantial contributions to conception and design, analysis and interpretation of data, and drafted the manuscript as well as revising it critically for important intellectual content. MV also gave final approval of the version to be published. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Athanasopoulou, C., Hätönen, H., Suni, S. et al. An analysis of online health information on schizophrenia or related conditions: a cross-sectional survey. BMC Med Inform Decis Mak 13, 98 (2013). https://doi.org/10.1186/1472-6947-13-98
- Consumer health information
- Information retrieval