Participatory methods in a digital setting: experiences from the co-creation of an eHealth tool for people with chronic obstructive pulmonary disease
BMC Medical Informatics and Decision Making volume 22, Article number: 68 (2022)
Using participatory methods to engage end-users in the development and design of eHealth is important to understand and incorporate their needs and context. Within participatory research, recent social distancing practice has forced a transition to digital communication platforms, a setting that warrants deeper understanding. The aim of this study was to describe the experiences of, and evaluate a digital co-creation process for developing an eHealth tool for people with chronic obstructive pulmonary disease (COPD).
The co-creation was guided by Participatory appreciative action and reflection, where a convenience sample (n = 17), including persons with COPD, health care professionals, relatives and a patient organization representative participated in six digital workshops. User instructions, technical equipment, and skilled support were provided if necessary. Workshops centred around different topics, with pre-recorded films, digital lectures and home assignments to up-skill participants. Process validity, experiences and ownership in the co-creation process were evaluated by repeated respondent validation, member checking, questionnaires and by assessing attendance. Data was analysed quantitatively or qualitatively as appropriate.
The co-creators were in general satisfied with the digital format of the workshops. Mean attendance and perceived engagement in workshops was high and the experience described as enjoyable. Engagement was facilitated by up-skilling activities and discussions in small groups. Few had used digital communication previously, and feelings ranging from excitement to concern were expressed initially. Technical issues, mainly audio related, were resolved with support. At completion, skills using equipment and digital platform surpassed expectations. Few disadvantages with the digital format were identified, and advantages included reduced travel, time efficiency and reduced infection risk.
Experiences of digital co-creation were overwhelmingly positive, despite initial barriers related to computer naivety and use of digital equipment and platforms. The high level of satisfaction, engagement, attendance rates, and agreement between individual and group views suggests that a digital co-creation process is a feasible method. Several important success factors were identified, such as the provision of information and education on discussion topics in advance of workshops, as well as the smaller group discussions during workshops. The knowledge gained herein will be useful for future digital co-creation processes.
A progressive digitalization of public health is emerging, with electronic health (eHealth) advocated as a means to support and enable public health without compromising quality, accessibility, efficiency, and equity . eHealth encompasses computers, smartphones, tablets, mobile sensors and wearables, apps, social media and online information, which can be used to personally monitor and inform health, as well as for communicative interaction, and the collection, management and utilization of health data . In particular, eHealth may facilitate greater control of, and a more active role in personal health ; its potential for self-management of chronic diseases, including diabetes, cancer, cardiovascular-, and respiratory disease across populations is promising [3, 4]. For effective self-management of chronic disease using eHealth products or systems, the understanding and incorporation of intended users’ needs and context appears to be a pre-requisite . For this purpose, the engagement of representatives of the specific population targeted (end-users) in the development and design of eHealth is increasingly promoted, and often approached using, for example, participatory methods or co-creation [6, 7]; the latter defined as ‘collaborative public health intervention development by academics working alongside other stakeholders .
Chronic obstructive pulmonary disease (COPD) is one of the most common chronic diseases, with substantial ensuing risk of morbidity and mortality . Pulmonary rehabilitation is a comprehensive intervention including, but not limited to, tailored physical exercise, education and behaviour change . Conclusive evidence suggests that pulmonary rehabilitation benefits patient-orientated outcomes in COPD such as improved physical capacity [11, 12]. Nevertheless, access to pulmonary rehabilitation is limited to a minority of the COPD population [13, 14], and improved access has been highlighted as a top priority for the coming decade. eHealth as a means to deliver pulmonary rehabilitation is a promising way to increase access, with increased reach , and level of physical activity indicated [16, 17]. However, recent systematic reviews of the effects of eHealth for the management of COPD have shown inconsistent results due to the high variability between studies and the interventions evaluated [18,19,20,21,22,23,24,25,26,27]. Larger studies, with high-quality study design, and where the eHealth intervention has been developed in co-creation with end-users, are warranted.
With the ambition to increase access to evidence-based treatment for people with COPD, our research group began preparations for a project to develop a novel eHealth tool, which later was named Min KOL (Eng: My COPD), in co-creation with prospective end-users and stakeholders through a series of workshops. Early in the project, the COVID-19 pandemic, and subsequent social distancing practice to protect risk groups, forced a transition of the entire co-creation process onto a digital communication platform. Digital co-creation has been suggested as an opportunity for wider inclusion and several potential advantages have been mentioned . Although digital co-creation processes are relatively uncommon within research to date, this will likely change in line with the ongoing digitalization of most areas of society and therefore requires deeper understanding. The aim of this study was to describe the experiences of, and evaluate a digital co-creation process for developing an eHealth tool for people with COPD.
The procedural components of the co-creation process for the eHealth tool adhered to recommendations and principles outlined by Leask et al. . The co-creation process was guided by Participatory appreciative action and reflection (PAAR). PAAR is a participatory method used to generate knowledge by exploring and developing success factors and facilitating collective learning, pluralism and reflection .
End-users and co-creators
A convenience sample of co-creators were approached and invited to participate in the digital co-creation process and recruited through the authors' professional networks, primary- and specialist care registers, and the national patient organization. Co-creators were defined as persons with COPD, that is, the end-users , but also stakeholders involved in the tool's later implementation and use. Criteria for eligibility as end-user co-creators were (1) a COPD diagnosis according to the Global initiative for chronic obstructive lung disease (GOLD) , (2) ability to use a smartphone or a tablet, and (3) living within public health care regions Västerbotten or Stockholm, Sweden. Further representatives of stakeholders perceived to be involved in the use and implementation of the eHealth tool, and who were invited to take part in the workshops as co-creators, were physiotherapists, a COPD-nurse and a physician experienced in pulmonary rehabilitation within primary care or specialized care, relatives (of both sexes), and a representative from the national patient organization. Five persons with COPD and one relative to a person with COPD declined participation due to the group setting (n = 1), unknown reasons (n = 2) and due to fears of not being able to master the digital communications platform and associated technical equipment (n = 3), while three of the health care professionals approached declined due to the added workload. The sample was composed with end-user co-creators constituting a majority and aiming for maximum variation to encompass a wide spectrum of perspectives , with regards to sex, age, disease severity, ethnicity, and urban or rural living. Researchers (SL, AT and PS) from Umeå University and Karolinska Institutet took part in and moderated the workshops. In the first workshop a software developer moderated one of the small discussion groups. The moderators (SL, AT, PS) had previous experiences of conducting qualitative interviews, co-creation workshops and/or digital teaching. The moderators designed the study, the content and structure of the workshops and home assignments based on guidelines for co-creation . Background data of co-creators, including age, sex, pulmonary function, symptoms using the COPD assessment test , ethnicity, and level of physical activity using questions from the Swedish National Board of Health and Welfare  were collected as appropriate (Table 1).
Digital video communication
The workshops were conducted through the digital video communication platform Zoom (Zoom Video Communications, Inc 2021). The co-creators took part in the digital workshops from a location of their choice. A simple user manual with written instructions on how to download, access and use the video communication platform was developed and provided, as well as technical equipment (headsets, tablets, web cameras, mobile surf) when needed.
Three technical meetings were hosted before workshops commenced, where co-creators were encouraged to access the platform and receive an introduction to basic functions required in the workshops, for example, how to turn on/off audio- and video functions, change the view and ask questions. Two skilled technical staff, from the department of information and communication technology (ICT) services and system development at Umeå University, hosted and took part in the technical meetings. The ICT-staff were also present at the beginning of each, and available between each workshop via e-mail and telephone, to solve any issues related to the technical equipment or to the digital communication platform.
Workshops, with discussions and practical tasks, are used within participatory methods to produce and collect reliable and valid domain specific data . The purpose of the workshops was to explore population specific features and requirements regarding content and design and to develop the eHealth tool accordingly. A series of 6 workshops were hosted by the researchers (Fig. 1). Each workshop lasted no more than 2 h and had regular breaks to optimize concentration and reduce risk of digital fatigue . The workshops were scheduled to start between 2–3 pm, since symptoms of COPD often present as less severe in the afternoon compared to the morning . The health care professionals and the patient organization representative were invited to participate in 3 workshops only (workshop 1, 3 and 5), as we anticipated that daytime meetings could hinder participation. Furthermore, upon own request, the patient organization representative also participated in the last workshop. The workshops were spaced 2–4 weeks apart to allow time for the moderators to prepare or follow-up on ideas and suggestions arisen in workshops or in evaluations, and to not cause an unnecessary burden on the co-creators. Between the workshops, the moderators also discussed how to facilitate engagement by the co-creators in the following workshops. All information and contact with co-creators occurred via telephone or e-mail, with information and films provided to co-creators in advance of workshops and placed on a shared drive accessible to all co-creators.
The workshops had an iterative structure, and contained reflection, activity, planning and evaluation. Except for the first, each workshop began with a short reflection whereby a summary from the previous workshop discussions and outcomes were presented and its content and accuracy refined by the co-creators. The agenda and aims of the day’s activities were outlined, which included presentations from invited speakers, as well as whole or small group discussions, which constituted a majority of the workshops. For the small group discussions (20 to 40 min) moderators divided the co-creators into 3–4 groups, which varied between workshops. Sex, age, type of co-creator and level of engagement was considered when allocating co-creators to groups to promote interaction. Towards the end of each workshop a plan for the next workshop was presented and corroborated with co-creators, including a home assignment that could comprise watching a film, reflecting on a particular question, or interviewing a friend or relative. After each workshop, all co-creators were asked to complete a digital questionnaire. Moderators completed a separate digital questionnaire related to the co-creation process.
Important aspects and components of co-creative design that were incorporated and encouraged were co-creator ownership to increase creativity and productivity . The equal standing of all co-creators, important for ownership, was affirmed and promoted by moderators in each workshop. Rules and responsibilities were negotiated by co-creators, for example, when and how to speak, and to respect each other’s opinions. Co-creators also took part in aspects of the planning process and decision making at workshops, for example, by viewing and commenting on the following workshop’s topic. Furthermore, all co-creators were “up-skilled”, i.e. received relevant information and knowledge necessary for meaningful participation and informed decision making in the workshops. By up-skilling, the capacity and capability of co-creators can increase  and potentially result in more innovative and meaningful solutions . Up-skilling was conducted by watching pre-recorded films or digital subject-area expert lectures, or by browsing specific web-based platforms for information in between and in preparation for workshops. Most up-skilling activities took place between workshops. This freed up time in the workshops for interactive activities and discussions that facilitated co-creator engagement. Activities in the workshops centred around pre-planned topics that were considered important for the development of the eHealth tool (Fig. 1). The initial workshop focused on introducing the co-creators and to get to know each other, with the aim to facilitate interactions in the digital workshops. A running topic throughout the workshops, was the creation of a name and a graphical element to represent the eHealth tool to enhance ownership.
Data on characteristics of co-creators were collected using structured questions during a phone call prior to the co-creation process.
All group discussions in the workshops were documented through audio recordings and notes taken by the moderators. Moderators kept a log of all further communication related to the project, including mail- and telephone conversations. An evaluation of process validity was embedded throughout the co-creation process to ensure that outcomes were representative of the co-creators’ opinions and suitable, tailored and valid for end-users . The evaluation was conducted by “respondent validation”, where within each workshop moderators’ summaries of the small group discussions were presented to the whole co-creator group and refined. The evaluation was also conducted by “member checking”, whereby the co-creators were asked to reflect on discussions and conclusions conducted in the previous workshop.
The co-creators’ experiences and ownership were evaluated through a digital questionnaire distributed at the end of each workshop, and at the completion of the workshop series. In the questionnaires at the end of each workshop, open questions were used to explore perceptions of the information provided and experiences of taking part, as well as, ways to carry across their views in the digital workshops. Furthermore, multiple choice questions were used to assess perceived agreement between own views and group conclusions (no-, part-, near complete-, complete agreement) and engagement in the workshops (not-, partly-, completely engaged). In the questionnaire at workshop series completion, experiences of the advantages and disadvantages of conducting the workshops digitally, the home assignments (up-skilling), and handling the technical equipment necessary were explored. Another aspect of ownership is loyalty [8, 39], which was evaluated by assessing the attendance rate of the co-creators in the study.
Moderators completed a separate digital questionnaire at the end of each workshop, and at the end of the workshop series. With the purpose to continuously develop and evaluate the co-creative process in the workshops, they answered questions related to facilitating factors and barriers, and the co-creators’ engagement in the co-creative process.
Co-creator characteristics, attendance and response rates were summarized using descriptive quantitative statistics. The co-creators’ responses in the questionnaires were summarized using descriptive qualitative analysis. First the responses were read through, and sorted into themes: experiences of the workshops, experiences of knowledge up-skilling and engagement, and experiences of the digital format of the workshops. Each theme was then described in text, with illustrative quotes from the evaluations. The same procedure was conducted for the moderators’ evaluations, notes and summaries related to the themes: the moderator role, and facilitators and barriers for engagement. The qualitative analysis was inspired by thematic analysis . The transcribed audio recordings were read through, and indications of technical issues noted. The qualitative and quantitative findings were interwoven and presented together in the results section. Two of the moderators analysed the quantitative (AT) and qualitative (SL) data and described the results, while the third moderator (PS) and two researchers (KW and AH) who were not involved in moderating the workshops critically reviewed the results.
All authors are registered physiotherapists, with extensive experience in research and clinical work within exercise training for people with COPD (PS, KW) or older adults (AT, AH). The authors also have experience working with and developing eHealth (SL, PS, AH, KW) and in the development of eHealth tools for people with COPD using co-creation (SL, KW). Finally, the authors have expertise conducting research studies utilising qualitative (SL, AH), quantitative (AT, KW, AH) or mixed methods (SL, KW, AH).
The study included 17 co-creators in total, 11 women and 6 men. The end-user co-creators were aged between 51 and 87 years, and were diagnosed as GOLD 2–4. Three participants receiving long-term oxygen treatment. All co-creators except one were born in Sweden. Characteristics of the co-creators are presented in Table 1.
The median attendance rate in workshops was 89% in persons with COPD, while remaining co-creators (relatives, health care professionals, patient organization representative) had 100% attendance rate. Reasons for not attending workshops were related to personal engagements such as attending a funeral or work related. In the last two workshops, most non-attendance occurred (n = 5), and reasons included computer failure (n = 1), forgetfulness (n = 1), acute illness (n = 1), unknown (n = 1), and deceased (n = 1).
General experiences of the process
The co-creators were in general satisfied with the structure and content of the co-creation process. Participation in the workshops was described as interesting, nice, informative and rewarding. The structure of the process was considered clear and varying, and varying the activities between larger and smaller groups was appreciated. The length of the workshops and the time between workshops were generally perceived to be adequate. However, a wish for more time for discussions in the smaller groups was expressed, since this setting was highlighted as particularly valuable for discussions. The information provided in the beginning of and between the workshops, including the films with pre-recorded lectures by the invited subject matter experts, was described as interesting, informative and clear.
Good with collated instructions and notes. Easy to look back on.
(Co-creator, final evaluation)
Although opinions initially varied in the discussions, the co-creators were able to come to an agreement on issues.
Interesting to see that we have such different starting points. But we nearly always reach the same end point.
(Co-creator, workshop 4)
Most co-creators rated a complete or near complete agreement between their individual views and the conclusions from the group discussions (Fig. 2), suggesting that process validity was high.
Experiences of shared knowledge
To get information about the aim of the workshops and the project, as well as about the expected results and future plans was considered important by the co-creators. A wish for more workshops to test the eHealth tool before its release was requested. The co-creators also expressed a desire to participate in more research studies in the future.
The up-skilling tasks and home assignments between the workshops was considered important for their engagement in the workshops.
I think it is important with some preparation with for example the film so that the workshop itself gets going easier.
(Co-creator, workshop 5)
In the co-creation process, it was the co-creators' experience that new knowledge about COPD was gained. New knowledge about aspects of pulmonary rehabilitation, especially the importance of physical exercise and how to perform it was mentioned. Information about COPD and advice for promoting health was appreciated, as well as new knowledge about which interventions they were entitled to.
The knowledge about COPD that I did not have before. I feel especially grateful for that and know more what I can ask for at the health care centre.
(Co-creator, final evaluation)
The perceived engagement in workshops was by most co-creators rated as high (completely engaged) throughout the workshop series (Fig. 3). The co-creators felt able to contribute with their opinions in the workshops and discussions. In addition, they found it interesting to engage with other people with knowledge of what living with COPD entails, and to share experiences and thoughts. The co-creators described the discussions as creative and constructive, that the communication in the group was good, Thus, the co-creators believed that they complemented each other and they seemed to enjoy getting to know each other. A respectful and relaxed climate in the group was according to the co-creators valuable for their engagement. To feel welcomed, seen and heard made expressing views easy.
We talked to each other and gave feed-back. Everyone talked in the small group and it was a good and friendly atmosphere. It was also easy to engage in the large group. If I thought something about an issue, I would express my opinion or idea.
(Co-creator, workshop 1)
The digital format of the workshops
In total, nine co-creators borrowed technical equipment, 2 web-cameras, 9 headsets, and 3 tablets. Mobile surf was supplied to one co-creator. While the health care professionals and patient organization representative had all used the video communication platform previously, of the end-user co-creators and relatives only one had previous experience.
The digital format of the workshops was in general experienced as positive by the co-creators. Nevertheless, prior to the first workshop, a few co-creators described feeling excited and a bit nervous about participating in the digital workshops, due to low computer skill and unfamiliarity with the digital technology.
New and a little gruesome before with the digital that I do not have much experience of.
(Co-creator, final evaluation)
However, as the workshops commenced, surprise and astonishment about how well they managed the digital technology was expressed.
Since I have no computer experience it was exciting to see if I would keep up. But it was just to talk.
(Co-creator, workshop 1)
Technical issues were noted during all workshops, in total on 51 occasions (5–11 per workshop). Most issues related to difficulties using technical equipment or functions in the digital platform, for example turning the sound on or off or positioning the microphone correctly (n = 17), using screen sharing or adjusting screen view (n = 12), moving into break-out groups or leaving the meeting (n = 4); and resolved in the workshop by moderators, co-creators or the ICT-staff. The technical issues between workshops concerned downloading and accessing the video communication platform and sound issues, which the ICT-staff resolved by telephone support. Still, a few major efforts were needed by the ICT-staff; one home visit and one remote control of a computer to download and install the video communication platform, and support to one participant by telephone and email to resolve persistent sound issues. In total, the ICT-staff provided 13 h of support throughout the digital co-creation process. Technology and support were also mentioned by the co-creators in the evaluations. The written instructions on how to install and use Zoom, and the option to borrow technical equipment was appreciated. As the workshops progressed, an increase in technical capability and confidence in participating in digital meetings were described.
Not many disadvantages of performing digital workshop were identified, although technical issues were mentioned. However, in comparison to digital workshops, real-life meetings were perceived to offer better possibilities to “get to know the others”, to “talk in a different way”, and “probably more fun”.
In contrast, several advantages of performing digital workshops were mentioned, such as avoiding travel and parking, reduced infection risk, time efficient, simple, and the possibility to participate regardless of geography.
Many participants and also invited “guests” in the same place, even though they were spread out over the country.
(Co-creator, final evaluation)
Development of the co-creative process
During the study, the moderators’ views of the importance of engagement in the co-creation process was affirmed. They perceived group belonging to be facilitated by the design of a graphical element for the eHealth tool. Technical issues, for instance, reduced sound, were viewed as the largest barrier for engagement. Further, the workshops were perceived to have an open and pleasant climate, where everyone got to speak. The moderators were responsible for noticing if a co-creator had been quiet for a while and to encourage inclusion in the discussion. It was experienced as stressful to both facilitate the discussion and take notes at the same time, however the co-creators assisted the moderators in noticing a problem or if someone wanted to say something. The moderators described the need for a different approach in digital workshops compared to real-life meetings. They viewed their role as moderators important in the digital workshops, especially in order to get a flow in the discussions.
Digital implementation makes it easier for participants to delay their answer or share their thoughts, the forum demands that the situation is more controlled.
(Moderator, workshop 5)
The moderators found it difficult to find balance in how much to structure the discussions in the digital format. In the initial workshops they perceived that much of the discussions in small groups occurred through them, and that the co-creators for the most part only answered questions addressed directly to them by the moderator, rather than openly discussing with each other. This was by the moderators viewed as a great responsibility.
It still feels a bit stressful that the group’s conversation depends a lot on me as leader, it is important to bring out the group’s opinions and experiences, you do not want to miss anything because you were not able to extract from the participants that which was important to them.
(Moderator, workshop 2)
It was therefore perceived that semi-structured questions or work tasks were needed to stimulate discussions, which would aid the moderators’ in facilitating the discussion by asking questions and distributing the word between co-creators. The moderators expressed a wish for discussions and interactions to be freer and more spontaneous, and when that happened the moderators’ role was perceived as more enjoying, easy and relaxing. The responsibilities were divided between moderators and kept constant throughout the co-creation process, which was experienced as positive and reassuring. As the moderators grew more comfortable in their roles they felt that their skills complemented each other’s.
Overall, it was the moderators’ experience that while all co-creators contributed, engagement was especially high in the smaller groups. Thus, the small group discussions were considered pivotal for engagement, while discussions were inhibited in the large group. Consequently, when planning the workshops, the smaller group discussions were prioritised and allocated sufficient time. The small group discussions were often described as slow to start, but became more active as the co-creators got more used to each other and the process.
Moderators viewed co-creators with more experiences related to the topic of the workshop as more active than co-creators with no or little experience. Consequently, well prepared workshops and clarity about the method, topics, home assignments and the structure of the workshops was described as important for engagement. The home assignments and question time with subject matter experts were seen as a facilitator for discussions. In addition, the moderators viewed the varied background and competence of the co-creators as an advantage that stimulated the discussions. However, early in the process it was perceived that the health care professionals held back in the discussions to allow the other co-creators room to talk.
The health care professionals seemed a little hesitant, as if they were primarily letting persons with COPD take their place.
(Moderator, workshop 1)
Consequently, when relevant for the topic, the smaller discussion groups were formed with health care professionals only in one group.
To the best of our knowledge, this is the first study to report experiences from a digital co-creation process in the development of an eHealth tool for people with COPD. The experiences were overwhelmingly positive. The co-creators were satisfied with the content and structure of the co-creation process, and the smaller group discussions were considered an especially valuable setting for discussions. While the workshops had good attendance rates throughout the workshop series, the last workshop seemed to have the lowest attendance rate. Engagement in workshops appeared high throughout and believed to be facilitated by preparations which included providing co-creators with information of the content in workshops and discussion topics in advance, and the creation of a respectful and relaxed climate in the workshops. Most co-creators reported a complete or near complete agreement between their individual views and the conclusions in the group at workshops. Some advantages of the digital format included avoidance of travel and parking problems, reduced infection risks, and the geographical diversity of participants. Barriers included computer naivety and limited experience of digital equipment and video communication platforms. Despite initial computer naivety and apprehension, the digital format of the co-creation workshops was experienced as positive, and the co-creators were surprised at how well they managed the technique after all. Some technical issues were noted throughout, by moderators and co-creators alike, which could be resolved in the majority of cases.
Interpretation of findings
Recruitment, support and training have been pointed out as key factors for meaningful involvement of patient and public representatives [8, 41]. Convenience sampling is the most reported method to identify patients in studies with patient engagement , and was used to recruit not only participants with COPD, but also relative-, health care-, and patient organization representatives in this study. In the recruitment of participants, a balanced socio-demographic representation is important to consider . We strived to include co-creators with varied perspectives, and believed that sex, age, disease severity, ethnicity, and urban or rural living, could influence preferences for an eHealth tool. Another aspect to consider in the recruitment process is the potential participants’ contribution to the group dynamic. Participants who domineer the discussions, or remain passive, may influence the group dynamics negatively . This study found, that in a digital setting the moderators had an important role in engaging the participants. The importance of a moderator, whose role is to allow everyone to be heard, has been confirmed elsewhere . A dedicated contact person who can answer questions between or before meetings has been suggested to support and facilitate engagement . In our study, one researcher was responsible for most email and telephone contacts with participants, and was therefore often contacted by the participants if questions arouse or if unable to attend a workshop. Sufficient time to build reciprocal relationships, for discussions [42, 43], and repeated meetings to build group dynamics  have also been reported important for engagement. In this study, although participants perceived the workshops to have an adequate time-frame, more time allocated for discussions, as well as more workshops were wished for. Meanwhile, at the two last workshops attendance was attenuated, possibly a result of decreased motivation. Motivation has been described as a driving force for people with COPD and health care professionals to participate in a co-creation process . Training, or up-skilling participants, has been reported as important for engagement and creativity in many studies [8, 41, 43, 45], which our results supports. In the process planning of this study, several steps were taken to facilitate knowledge development in the participants, such as information on the aim of the process and the participants’ role [43, 45], education in the topics of the workshops, and adequate time for preparations . According to Leask, et al. , up-skilling is an important part of ownership. In our study, ownership was further facilitated by promoting the equal standing of co-creators and moderators alike, and including co-creators in planning and decision making. The positive experiences by the co-creators and moderators suggest that ownership is an important feature in co-creation. In people with COPD and health care professionals, feelings of responsibility have been reported as a driving force in co-creation .
Computer literacy is an important part of eHealth literacy, and includes the ability to use computers to solve problems and to adapt to new technologies and software . The participants in this study experienced the digital format of the workshops as positive. In Sweden, more than 90% of the population in most age groups have access to the internet at home . Furthermore, in a cohort of people with COPD in Sweden, over 90% had access to the Internet and 85% almost used it every day . This provide good opportunities for conducting digital workshops. In groups with chronic respiratory diseases, positive views towards eHealth tools and high probabilities of using such tools have been shown [48, 49]. Still, in this study technical issues were considered a barrier for engagement. In the recruitment process, three potential participants declined participation in our study because of a fear to not master the digital format. Comfortability with digital tools have been shown to affect usage of digital tools in people with COPD . The co-creators with COPD in this study all owned and had used a smartphone or a tablet previously, however, the level of computer skills varied. Some had low computer skills and were unfamiliar with using email for communication and needed assistance, while others had higher computer skills and were accustomed to using Zoom or other digital communication platforms. A Swedish study of older persons with chronic diseases, reported a lack knowledge and fear of using new technology, but also an interest and a willingness to learn more about digital tools . Similar to the participants’ experiences in our study, a transition over time was shown in a telemonitoring study with people with COPD, where initial insecurities of using the technical equipment gradually eased as confidence grew . In that study, more supervised testing of the equipment was requested to improve confidence . This was implemented in our study, where opportunities to test the digital platform before the co-creation process started were provided. Another successful approach was having ICT-staff who provided ongoing support to participants with technical issues in the workshop. This allowed the workshop to continue without interruptions. The technical support would have been easier if all participants had been provided with and used the same equipment.
The lessons learnt in this study were many. Using recommendations for planning the co-creative process, with adaptions for a digital forum, seemed a useful strategy as participants appreciated the structure and content. Keeping the digital workshop sessions relatively short may require a greater need to prioritise interactive activities such as discussions in the workshops. We found pre-recorded films between workshops very effective for up-skilling the co-creators, albeit time-consuming to plan and prepare. In the workshops, most time was allocated for small-group discussions, which were also found to aid discussions and engagement. Moderators may need to take a more semi-structured approach, with clear questions and activities prepared, when moderating digital workshops compared to real-life workshops. Furthermore, to have access to information and conclusions between workshops were appreciated by the co-creators. The time and efficiency for the provision of technical support, for example guides on how to download and use programs, and opportunities to practice were aided by including ICT-staff in the project. With support even co-creators with low computer skills were able to participate in the workshops.
Strengths and limitations
The adherence to guidelines for conducting and reporting results from co-creation studies , and the use of PAAR is a strength of this study. By structuring and planning the study accordingly, aspects important to the quality of the co-creation process were incorporated. Between workshops, the moderators continuously evaluated and discussed ways to facilitate engagement in activities, home assignments and by using different group constellations. According to the PAAR approach, activities and responses were designed to elucidate and build on positive experiences and good examples . Triangulation  between authors also strengthen the trustworthiness of the study. During the planning of the study, it was decided that to reduce workloads of the health care professionals and the patient organization representative, they should only take part in three of the workshops. Nevertheless, at the end of the workshop series the patient organization representative requested permission to participate in the last workshop. Since the participant was deemed to contribute to the group discussions the request was approved. In the recruitment, a variation in perceptions were strived for, including the ethnicity of participants. However, only a limited variation in ethnicity was achieved, with only one participant born outside Sweden. The difficulty in finding and recruiting individuals born outside Sweden has been reported in a previous study . Including individuals with different ethnicities would have enriched the study and is recommended in future research on the subject.
In this study, the experiences of the digital co-creation process in the development of an eHealth tool for people with COPD were overwhelmingly positive, despite barriers related to computer naivety and use of digital equipment and platforms. The level of satisfaction, engagement and loyalty throughout the workshop series, as well as the level of agreement between individual and group views suggests that a digital co-creation process is a feasible method. Although many advantages with the digital process were identified, such as, avoidance of travel and risk of infection, highlighted barriers included computer nativity and limited experience of digital equipment and video communication platforms. Several important success factors were identified, such as the provision of information and education on discussion topics in advance of workshops, as well as the smaller group discussions during workshops. The knowledge gained herein will be useful for future digital co-creation studies.
Availability of data and materials
The data used during the current study are available from the corresponding author on reasonable request; however, it may only be shared in part (to ensure confidentiality).
Chronic obstructive pulmonary disease
- FEV1 :
Forced expiratory volume in one second
Global initiative for chronic obstructive lung disease
Information and communication technology
Participatory appreciative action and reflection
Odone A, Buttigieg S, Ricciardi W, Azzopardi-Muscat N, Staines A. Public health digitalization in Europe. Eur J Public Health. 2019;29(3):28–35.
Shaw T, McGregor D, Brunner M, Keep M, Janssen A, Barnet S. What is eHealth (6)? Development of a conceptual model for ehealth: qualitative study with key informants. J Med Internet Res. 2017;19(10):e324.
Hanlon P, Daines L, Campbell C, McKinstry B, Weller D, Pinnock H. Telehealth interventions to support self-management of long-term conditions: a systematic metareview of diabetes, heart failure, asthma, chronic obstructive pulmonary disease, and cancer. J Med Internet Res. 2017;19(5):e172.
Whitehead L, Seaton P. The effectiveness of self-management mobile phone and tablet apps in long-term condition management: a systematic review. J Med Internet Res. 2016;18(5):e97.
Thies K, Anderson D, Cramer B. Lack of adoption of a mobile app to support patient self-management of diabetes and hypertension in a federally qualified health center: interview analysis of staff and patients in a failed randomized trial. JMIR Hum Factors. 2017;4(4):e24.
Kujala S. User involvement: a review of the benefits and challenges. Behav Inf Technol. 2003;22(1):1.
Kayser L, Kushniruk A, Osborne RH, Norgaard O, Turner P. Enhancing the effectiveness of consumer-focused health information technology systems through ehealth literacy: a framework for understanding users’ needs. JMIR Hum Factors. 2015;2(1):e9.
Leask CF, Sandlund M, Skelton DA, Altenburg TM, Cardon G, Chinapaw MJM, De Bourdeaudhuij I, Verloigne M, Chastin SFM, GrandStand SS, et al. Framework, principles and recommendations for utilising participatory methodologies in the co-creation and evaluation of public health interventions. Res Involv Engagem. 2019;5:2.
Vogelmeier CF, Criner GJ, Martinez FJ, Anzueto A, Barnes PJ, Bourbeau J, Celli BR, Chen R, Decramer M, Fabbri LM, et al. Global strategy for the diagnosis, management, and prevention of chronic obstructive lung disease 2017 report, GOLD executive summary. Am J Respir Crit Care Med. 2017;195(5):557–82.
Spruit MA, Singh SJ, Garvey C, ZuWallack R, Nici L, Rochester C, Hill K, Holland AE, Lareau SC, Man WD, et al. An official American Thoracic Society/European Respiratory Society statement: key concepts and advances in pulmonary rehabilitation. Am J Respir Crit Care Med. 2013;188(8):e13-64.
Lacasse Y, Cates CJ, McCarthy B, Welsh EJ. This Cochrane Review is closed: deciding what constitutes enough research and where next for pulmonary rehabilitation in COPD. Cochrane Database Syst Rev. 2015;11:ED000107.
McCarthy B, Casey D, Devane D, Murphy K, Murphy E, Lacasse Y. Pulmonary rehabilitation for chronic obstructive pulmonary disease. Cochrane Database Syst Rev. 2015;2:CD003793.
Wadell K, Janaudis Ferreira T, Arne M, Lisspers K, Stallberg B, Emtner M. Hospital-based pulmonary rehabilitation in patients with COPD in Sweden—a national survey. Respir Med. 2013;107(8):1195–200.
Arne M, Emtner M, Lisspers K, Wadell K, Stallberg B. Availability of pulmonary rehabilitation in primary care for patients with COPD: a cross-sectional study in Sweden. Eur Clin Respir J. 2016;3:31601.
McNamara RJ, Dale M, McKeough ZJ. Innovative strategies to improve the reach and engagement in pulmonary rehabilitation. J Thorac Dis. 2019;11(Suppl 17):S2192–9.
Loeckx M, Rabinovich RA, Demeyer H, Louvaris Z, Tanner R, Rubio N, Frei A, De Jong C, Gimeno-Santos E, Rodrigues FM, et al. Smartphone-based physical activity telecoaching in chronic obstructive pulmonary disease: mixed-methods study on patient experiences and lessons for implementation. JMIR mHealth uHealth. 2018;6(12):e200.
Lundell S, Holmner Å, Rehn B, Nyberg A, Wadell K. Telehealthcare in COPD: a systematic review and meta-analysis on physical outcomes and dyspnea. Respir Med. 2015;109(1):11–26.
Janjua S, Banchoff E, Threapleton CJ, Prigmore S, Fletcher J, Disler RT. Digital interventions for the management of chronic obstructive pulmonary disease. Cochrane Database Syst Rev. 2021;4:CD013246.
Schulte MHJ, Aardoom JJ, Loheide-Niesmann L, Verstraete LLL, Ossebaard HC, Riper H. Effectiveness of eHealth interventions in improving medication adherence for patients with chronic obstructive pulmonary disease or asthma: systematic review. J Med Internet Res. 2021;23(7):e29475.
Kruse C, Pesek B, Anderson M, Brennan K, Comfort H. Telemonitoring to manage chronic obstructive pulmonary disease: systematic literature review. JMIR Med Inform. 2019;7(1):e11496.
Bonnevie T, Smondack P, Elkins M, Gouel B, Medrinal C, Combret Y, Muir JF, Cuvelier A, Prieur G, Gravier FE. Advanced telehealth technology improves home-based exercise therapy for people with stable chronic obstructive pulmonary disease: a systematic review. J Physiother. 2021;67(1):27–40.
Calvache-Mateo A, López-López L, Heredia-Ciuró A, Martín-Núñez J, Rodríguez-Torres J, Ortiz-Rubio A, Valenza MC. Efficacy of web-based supportive interventions in quality of life in COPD patients, a systematic review and meta-analysis. Int J Environ Res Public Health. 2021;18(23):25.
Jang S, Kim Y, Cho WK. A systematic review and meta-analysis of telemonitoring interventions on severe COPD exacerbations. Int J Environ Res Public Health. 2021;18(13):25.
Cox NS, Dal Corso S, Hansen H, McDonald CF, Hill CJ, Zanaboni P, Alison JA, O’Halloran P, Macdonald H, Holland AE. Telerehabilitation for chronic respiratory disease. Cochrane Database Syst Rev. 2021;1(1):CD013040.
Sul AR, Lyu DH, Park DA. Effectiveness of telemonitoring versus usual care for chronic obstructive pulmonary disease: a systematic review and meta-analysis. J Telemed Telecare. 2020;26(4):189–99.
Liu F, Jiang Y, Xu G, Ding Z. Effectiveness of telemedicine intervention for chronic obstructive pulmonary disease in China: a systematic review and meta-analysis. Telemed J E Health. 2020;26(9):1075–92.
Shaw G, Whelan ME, Armitage LC, Roberts N, Farmer AJ. Are COPD self-management mobile applications effective? A systematic review and meta-analysis. NPJ Prim Care Respir Med. 2020;30(1):11.
Beresford P, Farr M, Hickey G, Kaur M, Ocloo J, Tembo D, and Williams O (eds). COVID-19 and Co-production in health and social care research, policy and practice: volume 1: the challenges and necessity of co-production. 1st ed. Bristol University Press, 2021. http://www.jstor.org/stable/j.ctv1p6hqjs.
Ghaye T, Melander-Wikman A, Kisare M, Chambers P, Bergmark U, Kostenius C, Lillyman S. Participatory and appreciative action and reflection (PAAR)—democratizing reflective practices. Reflect Pract. 2000;9(4):361–97.
Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105–12.
Jones PW. COPD assessment test—rationale, development, validation and performance. COPD. 2013;10(2):269–71.
Olsson SJ, Ekblom O, Andersson E, Borjesson M, Kallings LV. Categorical answer modes provide superior validity to open answers when asking for level of physical activity: a cross-sectional study. Scand J Public Health. 2016;44(1):70–6.
Orngreen R, Levinsen K. Workshops as a Research methodology. Electr J e-Learn. 2017;15(1):70–81.
Williams O, Tembo D, Ocloo J, Kaur M, Hickey G, Farr M, Beresford P (eds). COVID-19 and co-production in health and social care research, policy and practice: volume 2: co-production methods and working together at a distance. 1st ed. Bristol University Press, 2021. http://www.jstor.org/stable/j.ctv1p6hqk9.
Tsiligianni I, Kocks JWH. Daytime symptoms of chronic obstructive pulmonary disease: a systematic review. NPJ Prim Care Respir Med. 2020;30(1):6.
Harwood T, Garry T. Co-creation and ambiguous ownership within virtual communities: the case of the Machinima community. J Consum Behav. 2014;13(2):148–56.
Baranick E, Baird A, Vinze A. An economic framework for transitioning to capacity building. Glob Public Health. 2015;10(1):15–27.
Fischer G, Giaccardi E. Meta-design: a framework for the future of end-user development. In: Lieberman H, Paternò F, Wulf V (eds) End user development. Dordrecht: Springer. 2006: 427–457.
Roser T, Samson A, Humphreys P, Cruz-Valdivieso E. Co-creation: new pathways to value: an overview. Promise LSE Enterp. 2009.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.
Boivin A, Currie K, Fervers B, Gracia J, James M, Marshall C, Sakala C, Sanger S, Strid J, Thomas V, et al. Patient and public involvement in clinical guidelines: international experiences and future perspectives. Quality Saf Health Care. 2010;19(5):e22.
Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, Brito JP, Boehmer K, Hasan R, Firwana B, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14(1):89.
Armstrong MJ, Mullins CD, Gronseth GS, Gagliardi AR. Recommendations for patient engagement in guideline development panels: a qualitative focus group study of guideline-naïve patients. PLoS ONE. 2017;12(3):e0174329.
Kjellsdotter A, Andersson S, Berglund M. Together for the future—development of a digital website to support chronic obstructive pulmonary disease self-management: a qualitative study. J Multidiscip Healthc. 2021;14:757–66.
Cluzeau F, Wedzicha JA, Kelson M, Corn J, Kunz R, Walsh J, Schünemann HJ. Stakeholder involvement: how to do it right: article 9 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report. Proc Am Thorac Soc. 2012;9(5):269–73.
Norman CD, Skinner HA. eHealth Literacy: essential Skills for Consumer Health in a Networked World. J Med Internet Res. 2006;8(2):e9.
Statistics Sweden. Ict usage in households and by individuals. 2020; https://www.scb.se/en/finding-statistics/statistics-by-subject-area/living-conditions/living-conditions/ict-usage-in-households-and-by-individuals/. Accessed May 21, 2021.
Sönnerfors P, Skavberg Roaldsen K, Ståhle A, Wadell K, Halvarsson A. Access to, use, knowledge, and preferences for information technology and technical equipment among people with chronic obstructive pulmonary disease (COPD) in Sweden. A cross-sectional survey study. BMC Med Inform Decis Mak. 2021;21(1):185.
Seidman Z, McNamara R, Wootton S, Leung R, Spencer L, Dale M, Dennis S, McKeough Z. People attending pulmonary rehabilitation demonstrate a substantial engagement with technology and willingness to use telerehabilitation: a survey. J Physiother. 2017;63(3):175–81.
Marklund S, Tistad M, Lundell S, Östrand L, Sörlin A, Boström C, Wadell K, Nyberg A. Experiences and factors affecting usage of an ehealth tool for self-management among people with chronic obstructive pulmonary disease: qualitative study. J Med Internet Res. 2021;23(4):e25672.
Nymberg VM, Bolmsjo BB, Wolff M, Calling S, Gerward S, Sandberg M. “Having to learn this so late in our lives…” Swedish elderly patients’ beliefs, experiences, attitudes and expectations of e-health in primary health care. Scand J Prim Health Care. 2019;37(1):41–52.
Lundell S, Modig M, Holmner Å, Wadell K. Perceptions of home telemonitoring use among patients with chronic obstructive pulmonary disease: qualitative study. JMIR mHealth uHealth. 2020;8(6):e16343.
Dahlgren L, Emmelin M, Winkvist A, Lindhgren M. Umeå universitet. Institutionen för folkhälsa och klinisk medicin. Epidemiologi: Qualitative methodology for international public health. Umeå: Epidemiology and Public Health Sciences, Department of Public Health and Clinical Medicine, Umeå University; 2004.
Lundell S, Wadell K, Wiklund M, Tistad M. Enhancing confidence and coping with stigma in an ambiguous interaction with primary care: a qualitative study of people with COPD. COPD. 2020;17(5):533–42.
The authors would like to thank all the participants who contributed in the co-creation process, and COPD nurse Eva Sunna, oxygen nurse Ingegerd Karlsson and Docent André Nyberg who helped with the recruitment of participants.
Open access funding provided by Umea University. This research was supported by The Swedish Heart and Lung Foundation (Dnr: 20190406), The Strategic Research Area—Caring Science at Umeå University and Karolinska Institutet, Doctoral School in Health Care Sciences at Karolinska Institutet and King Gustav the 5th and Queen Victorias Freemason Foundation. The funders had no role in the design of the study, data collection and analysis, writing of the manuscript, or the decision to publish.
Ethics approval and consent to participate
The study has been performed in accordance with the Declaration of Helsinki and was approved by the Swedish Ethical Review Authority (Dnr. 2020-01693). Signed informed consent was collected from all participants before the start of the study.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Lundell, S., Toots, A., Sönnerfors, P. et al. Participatory methods in a digital setting: experiences from the co-creation of an eHealth tool for people with chronic obstructive pulmonary disease. BMC Med Inform Decis Mak 22, 68 (2022). https://doi.org/10.1186/s12911-022-01806-9