Table 1 Comparison of the four quality assessment tools based on the original tool for websites
Websites | PDAs | QPLs | Videos |
|---|---|---|---|
Content criteria | |||
Expertise | Item split in two, referring authors ‘ qualifications and inclusion of patients’ perspective; both assigned to formal criteria | Item refers to experts and potential users (physicians and patients) – assigned to formal criteria | No explicit item included |
Objectives (Are they evident? Are they being met? Is the target group specified?) | Item referring only target group | Item carried over with small changes | Item split in two, referring objectives and target group – the latter is assigned to user-oriented criteria |
Is the writing balanced and unbiased? | Item adapted referring pros and cons of treatment/screening methods | Item carried over | No explicit item included |
Precision (Are statements and information precise?) | No explicit item included | Item carried over | Item carried over with added aspect of numerical data/figures being given |
Relevance (Are information and illustrations relevant for users? Is the context of presented data relevant for users/target group?) | No explicit item included | Item carried over | Item carried over and assigned to user-oriented criteria |
Complementarity (Does the publication support and facilitate shared decision-making?) | Item changed: different ways of reaching a decision are made explicit | Item changed: objective to support, not to replace a consultation | Item changed: no replacement for consultation |
Standards of information (scientific evidence, up to date) | Item assigned to formal criteria; specified to communicating the quality of scientific evidence, including a lack of evidence | No explicit item included [not applicable to QPLs] | Item specified to evidence-based information |
No statements regarding topics without ascertained information (Does the publication comment on areas for which no ascertained information is available?) | No explicit item included | No explicit item included [not applicable to QPLs] | No explicit item included |
Detailed information about treatment procedures (Does the publication describe the benefits of each treatment procedure, the mode of action of each treatment procedure, the risks of each treatment procedure, possible consequences of non-treatment, how the treatment procedures affect quality of life?) | Item split in two | No explicit item included [not applicable to QPLs] | Item carried over |
Does the publication contain details of supplementary aids and information? | No explicit item included | Two items, referring additional information and the indication, that there are sources of information beyond the consultation with the physician | No explicit item included |
Statements are based on patient-relevant endpoints: Improvement in health status, reduction in the duration of the disease, prolongation of life, reduction of side effects | Similar item included in formal criteria (evidence from studies populations which are similar to the target group) | No explicit item included | No explicit item included |
Are the illustrations appropriate/understandable? (Additions of appropriate graphical representations are useful) | Item specified: illustrations support content without distracting | Item specified: illustrations, if used, support information processing | Item specified: illustrations adequately support verbal information |
Consideration of layout aspects | Item extended: layout aspects support the content and present positive and negative aspects of all options in a neutral manner | Item split in two: the formatting (font, font size) is easy to read + form of presentation does not grade the different items/topics | No explicit item included |
Communication of risks (More than a general, verbal presentation of risks. Presentation of potential loss and gain side by side) | Item assigned to formal criteria; specified regarding target population and consistent scale | No explicit item included [not applicable to QPLs] | No explicit item included |
Quality assurance procedures | Item assigned to formal criteria; specified to include professionals and lay users | Item assigned to formal criteria; specified to include professionals and lay users | Item assigned to formal criteria |
Clear arrangement of information (Is the information clearly presented? Is there a search function?) | Item carried over | Item split in two: clear structure + comprehensible order of items | Item split in two: clear structure + order of information facilitates understanding |
Completeness of information | Item separated into all relevant categories | Item carried over (all relevant topics listed in item description) | Item carried over |
Lack of evidence is openly communicated | Item assigned to formal criteria | No explicit item included [not applicable to QPLs] | Item carried over |
A user guide is included | |||
No repetition/redundancy among items | |||
Formal criteria | |||
Use of language that supports participation and is adapted to the target group | Item split in two and assigned to content criteria: language understandable for target group + guiding through the decision process to reach an individual decision | Item split in three, assigned to content criteria: language understandable for target group + encouragement to discuss option with physician + guiding through the conversation to attain all relevant information | Item assigned to user-oriented criteria |
Transparency (Are authors and data sources of the information named? Information about the provider? Disclosure of funding/sponsors? Disclosure of advertising policy? Separation of advertising and editorial content?) | Item split in three: developer information + funding + potential gains and losses of developers | Item split in four: developer information + funding + potential gains and losses of developers + clear separation of content and advertising | Item carried over |
Data protection (Is there information on data security? How is personal data used and protected) | Item changed: security for personal information entered into the website | Item split in two: explicit information on data use and protection + security for personal information entered into the website | No explicit item included |
Information on sources (Is there a clear indication of the sources of information used to produce the publication (in addition to the author or producer)? Is it clearly stated when the source material was produced?) | Item split and extended to four: scientific evidence cited + description of evidence quality (incl. lack of evidence) + additional information on selection process of cited evidence + last update of PDA given | Item changed: indications on up-to-dateness + steps for selecting question by creators are explained | Item split in two |
Scientific knowledge about the presentation of numbers and outcomes is taken into account | Item split in two and specified: probabilities correctly described as uncertain and put into context with other events + description of how probabilities have been calculated | No explicit item included | Item carried over and assigned to content criteria |
User involvement (opportunity for feedback from users; patients are involved in the process of information production) | No explicit item included | Item carried over | Item carried over and assigned to user-oriented criteria |
Low-threshold access for all patients within the target group without additional costs | Low-threshold access for all patients within the target group without additional costs | ||
Offers options other than reading (e.g. audio, video or in-person discussion) | Offers options other than reading (e.g. audio, video or in-person discussion) | ||
for web-based material | |||
Good navigation options integrated into the website | Good navigation options integrated into the website | ||
Easy to return to after linking to other websites | Easy to return to after linking to other websites | ||
Offers search for keywords | Offers search for keywords | ||
Can be printed as single document | Can be printed as single document | ||
Offers feedback on personal health information where they can be included | Offers feedback on personal health information where they can be included | ||
Easy to find online | |||
Medium-specific criteria | |||
Helps to recognise that a personal decision has to be made | Points out that users can make their own selection of items/questions | Video description (Do title and content match? Does the description give a summary of the video content? Is the focus of the video recognisable?) | |
Helps to know about the different options and asks patients to think about which positive and negative features of the options are most important to them | Points out that the formulations for the discussion with the doctor can be individually adapted and changed, and gives examples | Feasibility (Does the video clearly identify at least one action that users can perform? Does the video speak directly to users when describing the activity? Does the video break down each activity into manageable steps?) | |
Explains that personal values can influence decisions | Audio quality (Background noise avoided? Supporting background music, signal tones or similar? Volume appropriate? Speed of speech appropriate? Speaker easy to understand?) | ||
Improves the match between the options finally chosen and the values and characteristics that are most important to the patient | Visual quality (Objects and people well lit? Image sharp? Camera movement appropriate? Text in video easy to read? Visual material easy to recognise?) | ||
Provides guidance in formulating values relevant to the decision | |||
Describes both options of letting specific values impact the decision or not | |||
Helps to implement each step in relation to the decision form chosen | |||
Describes different aspects of options to help patients imagine what it is like to experience the physical, emotional and social impact of doing so | |||
If other patients’ stories/testimonials are included: | Stories describe both positive and negative experiences | ||
Stories enable relating to steps others have made in their decision-making process | |||
Stories introduce listed option and possible outcomes | |||
Describes steps to select the stories and to verify contained information | |||
Statement on informed consent included | |||
Disclosure of financial or other reasons for patients to have shared their stories | |||