Table 5 Requirements related to PP in the MPLC

1. General requirements

 • Recognition of the value of PP among stakeholders [24, 25, 39, 51, 59, 64]

 • Consensus on the role of PP in decision-making [1, 62]

 • More familiarity among stakeholders with PP studies [19, 34, 48, 66, 70, 72]

 • More educated researchers in preference research [53]

 • Resources to evaluate PP [1, 48]

 • Taxonomic work for PP research [1, 60]

 • Guidance on:

  ○ When during development to measure PP [1, 34, 51]

  ○ Which preference method to use in which circumstance [1, 40, 44, 56, 72]

  ○ Whose preferences to measure (e.g. required disease experience) [1, 19, 44]

  ○ Sample size [37]

  ○ Good research practice and quality criteria for PP studies [1, 19, 38, 43, 44, 62]

  ○ How to ensure validity of a PP study [75]

  ○ How to report about PP studies [44]

 • Further research to:

  ○ Validate and test preference methods [37, 44, 46, 66, 76]

  ○ Identify methods for integrating clinical evidence in PP study analysis [50, 56]

  ○ Investigate methodological issues (e.g. hindsight bias) [62]

  ○ Compare the performance of different methods in a given situation [37]

  ○ Determine impact of changing list of attributes with any given method [37]

  ○ Explore statistical methods to detect preference heterogeneity [77]

  ○ Guide the development of newer methods for eliciting PP [76]

  ○ Assess comprehension differences by participants between methods [76]

  ○ Assess impact of the level of previous education on PP [33]

  ○ Quantify the effect of the attribute descriptions on elicited PP [78]

2. Operational requirements

 • Requirements related to timing of PP study:

  ○ Decision depends on level of information of the treatments’ key risks [19]

  ○ Timing needs to be decided by sponsor [19]

  ○ During marketing phase to assess long-term side effects and burden [1]

 • Requirements related to dealing with PP study results:

  ○ Stakeholders should be prepared for disappointing PP study results [24]

  ○ PP study results should be provided to patient community and public [24]

  ○ Presentation of PP study results should be tailored to the audience [79]

  ○ PP study results should be described transparently [56, 75]

3. Quality requirements

 • General requirements regarding design, set-up and conduct of PP studies:

  ○ Selected research question should be answerable with PP study [75]

  ○ Study objectivity throughout PP study [24]

  ○ Independent design as design can influence analysis outcomes [25]

  ○ Extensive and forward planning [19, 27, 48]

  ○ Determination of objectives and attributes before design [24]

  ○ Design based on prior literature and preference information [19]

  ○ Clear definition of the patient sample and characteristics [19, 24, 49]

  ○ Training partners on methodology, objectives and expectations of study [79]

  ○ Good communication and documentation of changes to study plans [79]

  ○ Methodological expertise when designing and executing a PP study [24, 70]

  ○ Multi-stakeholder partnerships (patients, academics, industry) [24, 37, 79]

  ○ Interaction between decision-makers and industry in design [14, 19, 24]

  ○ Involvement of patients, caregivers and patient organizations [24, 42, 49, 51]

  ○ Application of ‘good science’ principles [1, 19, 24, 51]

  ○ Consideration of patient heterogeneity and cognitive burden [14, 40, 58, 75]

  ○ Consideration of internal and external validity [75]

  ○ Administration of survey by trained researchers [14]

  ○ Provision of tutorial for participants if self-administered survey is used [14]

  ○ Training of participants in elicitation tasks [40]

  ○ Ensuring participants’ understanding of aim and how results will be used [40]

  ○ Consideration of low level of health numeracy in general population [43]

 • Sample requirements:

  ○ Sample should be heterogeneous (large samples, setting quotas) [19, 49, 75]

  ○ Sample should be representative of population of interest [14, 19]

  ○ If not possible to elicit from patients, include proxies [19, 34]

  ○ Sample ideally is clinical trial population [71]

  ○ Sample ideally is broader population than clinical trial population [41]

  ○ Patient should be the focus, not health care professional [14]

  ○ Sample should be representative of affected patients [56]

  ○ Sample should be representative of target population [75]

  ○ Sample that can yield reliable results should be drawn [24]

  ○ PP should come from the same population as data of effectiveness [1]

  ○ Both patients in remission as well as patients in recovery should be included [50]

  ○ Sampling should consider sociodemographic and disease characteristics [50, 61]

 • Sample size requirements:

  ○ Adequate size so that results are generalizable to population of interest [14]

  ○ Sufficient size to generate acceptably robust results [24]

  ○ If subgroups: sufficient number in each subgroup [14]

 • PP results requirements:

  ○ Type of PP (qualitative vs quantitative) depends on stage and decision-making context of MPLC [1, 14, 16, 19, 60]

  ○ Type of PP should be determined by research question [19]

  ○ Clinical data should be collected and used to augment PP data [42, 43]

  ○ Patient’s willingness and unwillingness to accept risks should be measured [14]

 • Preference method requirements:

  ○ Method should be selected based on factors [1, 19, 40, 44, 76, 80]

  ○ Method should adhere to utility theory [18, 76]

  ○ Method should account for patient-relevant attributes/outcome measures [18]

  ○ Methods should be easy and simple for patients to understand [18]

 • Requirements regarding attribute selection:

  ○ Research question should guide attribute and level selection [75]

  ○ Attributes should be broader than clinical attributes to elicit meaningful trade-offs [41]

  ○ Attributes should be patient-centered to investigate meaningful attributes [49]

  ○ Attributes should come from existing clinical trials [50, 81]

  ○ Selection by literature, qualitative study, asking group of medical experts or decision-makers [19]

  ○ Patient representatives, patients and experts should inform selection [50, 62]

  ○ Attributes should not overlap [30, 35, 50]

 • Requirements regarding survey instrument:

  ○ Survey should be developed with input from multiple stakeholders [24]

  ○ Survey should be piloted [24, 40]

  ○ Survey should include screening questions, informed consent provisions, background information, training and definitions, testing, survey questions, follow-up survey questions [24]

  ○ Benefit descriptions and effectiveness measures should be carefully defined [78]

  ○ Patients should understand objective of the elicitation tasks and how data will be used [40]

  ○ Questions have to be asked in an open and understandable way [18, 56, 75]

  ○ For choice-based preference measures, options should:

   ■ Be clearly described [56]

   ■ Have realistic advantages and disadvantages [56]

   ■ Be communicated to patients together with their characteristics [80]

 • Requirements regarding the analysis:

  ○ Interpretation of results should consider the mode of sampling [68]

  ○ Interpretation of study results should be validated with patients [40]

  ○ Results should be considered with preferences from other stakeholders (clinicians, decision-makers) [68]

  ○ Appropriate stakeholders should interpret analysis [79]

  ○ Sources of uncertainty should be reported through confidence interval and/or standard error [14]

  ○ Written agreements about intellectual property and data use are needed [24]