Table 2 Steps in the “Planning” phase of the DECI evaluation planning

Choosing a methodological approach

To select a methodological approach for the evaluation

Members of the interdisciplinary research consortium prescribed to different research traditions; therefore, the methodological approach was a subject for discussion. Quantitative and mixed-method approaches were considered. The mixed-method approach was selected

Defining evaluation questions

To define the research questions and endpoints

The questions of evaluation defined in the project contract were revised through recurring communications among partners. Owing to the interdisciplinary nature of the consortium, differences in the preferred endpoints and evaluation questions were resolved and a final list of research questions, and primary and secondary endpoints was created

Defining outcome measures

To define the outcome measures that will help assess the research questions and endpoints

Common outcome measures among all project locations were preferred. After rounds of negotiations, common outcome measures were selected. Means of acquiring data on the measures were defined, which resulted in several standard quantitative measures to assess health outcomes, tailor-made surveys on willingness-to-pay and physical activity levels, qualitative interview protocols to cover perceptions of patients and staff, and MS Excel files for the collection of organizational process data

Planning data collection

To make arrangements for data collection

Each partner’s technical and organizational possibilities for data collection were studied, taking into consideration data security principles and privacy laws. A schedule, format, and medium of data collection and its storage were planned

Planning the monitoring of data collection

To ensure the quality of the data collected

A template for monthly reporting of the data collected was created and shared with the partners responsible for the data collection

Considering methods of data analysis

To plan the data analysis

Feasible methods of data analysis were discussed among the project partners. Resources, timeplan, and competences needed were considered. Responsibilities for different parts in the data analysis were shared among different partners

Defining expected results

To define the expected outcomes of the intervention

Related studies and the stakeholders’ opinions were taken into account when setting the expected quantitative values or qualitative goals for the evaluation

Defining a quality management plan

To define an action plan to monitor progress and quality of the project outputs

A plan for managing the quality of the project results was prepared as part of the project management activities. Bi-weekly calls and reporting among the partners were agreed upon