Details of development of the resource for adults with asthma in the RAISIN (randomized trial of an asthma internet self-management intervention) study

Abstract

Background

Around 300 million people worldwide have asthma and prevalence is increasing. Self-management can be effective in improving a range of outcomes and is cost effective, but is underutilised as a treatment strategy. Supporting optimum self-management using digital technology shows promise, but how best to do this is not clear. We aimed to develop an evidence based, theory informed, online resource to support self-management in adults with asthma, called ‘Living well with Asthma’, as part of the RAISIN (Randomized Trial of an Asthma Internet Self-Management Intervention) study.

Methods

We developed Living well with Asthma in two phases.

Phase 1: A low fidelity prototype (paper-based) version of the website was developed iteratively through input from a multidisciplinary expert panel, empirical evidence from the literature, and potential end users via focus groups (adults with asthma and practice nurses). Implementation and behaviour change theories informed this process.

Phase 2: The paper-based designs were converted to a website through an iterative user centred process. Adults with asthma (n = 10) took part in think aloud studies, discussing the paper based version, then the web-based version. Participants considered contents, layout, and navigation. Development was agile using feedback from the think aloud sessions immediately to inform design and subsequent think aloud sessions. Think aloud transcripts were also thematically analysed, further informing resource development.

Results

The website asked users to aim to be symptom free. Key behaviours targeted to achieve this include: optimising medication use (including inhaler technique); attending primary care asthma reviews; using asthma action plans; increasing physical activity levels; and stopping smoking. The website had 11 sections, plus email reminders, which promoted these behaviours. Feedback on the contents of the resource was mainly positive with most changes focussing on clarification of language, order of pages and usability issues mainly relating to navigation difficulties.

Conclusions

Our multifaceted approach to online intervention development underpinned by theory, using evidence from the literature, co-designed with end users and a multidisciplinary panel has resulted in a resource which end users find relevant to their needs and easy to use. Living well with Asthma is undergoing evaluation within a randomized controlled trial.

Background

Asthma is common, affecting 300 million people worldwide, and its prevalence is increasing; an estimated additional 100 million people will have asthma by 2025 [16]. Despite increasing costs associated with asthma [2], care remains suboptimal, with many patients tolerating symptoms and lifestyle limitations unnecessarily, due to suboptimal use of proven available therapies [11, 26, 29, 32].

Supporting optimum self-management by providing relevant self-management education including how to use an asthma action plan (AAP), regular health professional review, and optimal use of medications has been shown to have positive effects on a range of asthma outcomes such as improved quality of life, lower rates of healthcare contacts, and fewer days off work and school [7]. Promoting self-management is a recommendation in worldwide asthma guidelines [4, 8]. Despite this, self-management as a treatment strategy remains underused [8, 32, 34]. Recently, new information and communication technologies (ICTs) have been proposed as a means to improve asthma self-management uptake. We conducted a meta-review [22] which showed that online interactive resources to support self-management of asthma can be safe and effective at improving some outcomes such as markers of self-care, activity limitation, quality of life and medication use. However, interventions were poorly described and it was impossible to extract generalisable lessons about the key ‘active ingredients’ of interventions. This challenge has previously been recognised by Michie et al. [20] who in response have developed a taxonomy of behaviour change techniques (BCTs) which they propose researchers can use to describe interventions, in order to overcome the lack of a systematic way of determining the ‘active ingredients’ of a complex interventions [21].

Traditionally, development of online interventions has been resource intensive with each intervention requiring to be programmed individually by a team of programmers from scratch—a barrier to internet based interventions being cost-effective when compared to face-to-face or paper alternatives. A team in Southampton who have recognised the increasing potential for delivering health care online have developed an open access software package called LifeGuide [10, 38, 41]. LifeGuide aims to allow researchers from a non-computer programming background to more easily and flexibly create and modify internet-delivered interventions. It has been used successfully in a number of health related interventions [3, 33, 38]. A key design feature of LifeGuide is that it allows researchers to easily test parts of an intervention and immediately modify and improve it based on the findings, and to trial it in the development phase.

Here we describe the development and optimisation of the Living well with Asthma website which we developed using the LifeGuide open access software package guided by the updated MRC guidance for the development and evaluation of complex interventions [18]. The Living well with Asthma website is currently undergoing evaluation in a randomised controlled trial (RCT) called RAISIN (Randomized Trial of an Asthma Internet Self-Management Intervention) (ISRCTN 78556552) [23].

Methods

To develop the Living Well with Asthma website we followed the steps outlined in the updated MRC guidance which recommends that intervention development should be systematic, include review of the evidence, be theory based, and incorporate feasibility or user testing [18]. In this section we describe the two phases of work we undertook to incorporate these steps. Ethical approval was granted from the West of Scotland Research Ethics Committee (12/WS/0068), and all participants provided informed consent.

Phase 1: intervention planning

Phase 1 describes the process of developing a ‘first draft’ of the website. This phase consisted of three main work packages (WP), all overseen by a multidisciplinary ‘expert panel’ made up of 3 general practitioners, a respiratory physician, a health psychologist, a social scientist, and a human-computer-interaction researcher.

Work package 1—understanding the evidence & incorporating theory (scoping review and expert panel)

Campbell et al. [5] describe 5 key tasks involved in defining and understanding the ‘problem’ that your intervention is aiming to solve, namely: 1) defining and quantifying the problem; 2) identifying who is mostly likely to benefit; 3) understanding the pathways which contribute to the problem; 4) consideration of whether (and how) these pathways are amenable to change; 5) and attempting to quantify the potential for improvement. We did a brief scoping review of the literature and used the experience of our expert panel to work through these tasks. We identified a list of features that a resource should have, incorporating recommended behaviour change concepts [25].

Work package 2—getting user perspectives on a web resource (focus groups)

In order to investigate the plausibility of this list with potential end users we convened 2 focus groups, consisting in total of 9 adults with asthma (6 female, 3 male), and 4 practice nurses who undertake asthma reviews. Recruitment was undertaken using a range of sources: primary care, Asthma UK Research and Policy volunteers, Chest Heart Stroke Scotland volunteers and a secondary care asthma clinic. Adults aged 18 and over were eligible provided they had a diagnosis of asthma and could provide informed consent. There was no upper age limit on participation. Participants could agree to participate in a focus group, up to two think aloud studies (described in phase 2), or both. Focus groups were held at the Department of General Practice & Primary Care, University of Glasgow, and were audio recorded and transcribed. We used the implementation theory Normalization Process Theory (NPT) [24] to inform the topic guide for these focus groups (Fig. 1). This theory was used as it is being increasingly advocated as a means to understand implementation processes and enhance the implementability of interventions [17, 24]. Discussion focused on the perceived barriers and facilitators to sustained use of an online resource to support self-management. This consideration of implementation issues at such as early stage is a key message from the MRC guidance. Our list of potential features derived from WP 1 was explored using questions generated from our topic guide (Fig. 1) and we sought suggestions for additional features. The focus groups were transcribed and any statements which were barriers or facilitators to self-management were extracted, along with any suggestions for features to include in our website.

Fig. 1

Focus group topic guide (NPT informed)

Work package 3—developing a draft version of the website (expert panel)

Using information gathered from WP 1 and 2, the list of suggested features to include in the website was reviewed and refined iteratively. As low fidelity prototype pages were generated (initially using Microsoft Word or PowerPoint) (also referred to as draft pages) they were reviewed initially by those in the panel with a clinical background to ensure the content was factually correct. Subsequently the pages were shown to members of the expert panel with specific expertise in behaviour change theory to ensure maximum opportunity for promoting behaviour change was incorporated into each page or section, using the behaviour change concepts agreed on from WP1 [25]. From this a draft version of each potential webpage was finalised, ready for think aloud study evaluation.

Phase 2: iterative refinement of the resource contents of the website (think aloud studies and expert panel)

Draft pages developed at the end of Phase 1 were gradually translated into interactive webpages with input from potential end users in the form of think aloud studies, and review by the expert panel. While LifeGuide can be used by researchers with no computing science background, due to time constraints, a programmer transferred the majority of the draft pages into LifeGuide initially. Think aloud studies were undertaken by the first author at either the participant’s home, or the Department of General Practice & Primary Care, University of Glasgow, depending on individual participant preference. Participants were recruited from the same pool as the focus groups, and they could participate in a maximum of two think aloud studies. There were two waves of think aloud studies: the first 4 used draft webpages still on paper or PowerPoint slide, the latter 6 were undertaken completely using the prototype webpages on LifeGuide. Participants were asked to say whatever they thought or felt about what they were seeing, with prompts and questions used to elaborate on responses. The participants were then encouraged to voice any additional suggestions or opinions to improve the resource, for example what they liked and disliked, what was intuitive and what was not, and how they envisaged using such a website in real life in the future. The majority of the findings from the think aloud studies were acted upon immediately after the session by the researcher doing the think aloud studies, in order to progress the resource ready for the subsequent think aloud study. We also thematically analysed the transcriptions of the think aloud studies with the aim of providing information for further development of the resource following the pilot RCT. Thematic analysis was undertaken using a coding frame developed by DM. Both DM and SW independently coded the first 2 transcripts, and results compared, after which DM coded the remaining transcripts. Comments were noted to be either a positive comment, where the user liked or identified with what they saw, or a negative comment where the user disliked or disagreed with what they saw, or where the user suggested an improvement or alternative way of presenting the data. The final version of the Living well with Asthma website was formally mapped to Michie and colleagues latest BCT taxonomy [21] in order to describe which BCTs were present. Every page of the website was reviewed by the first author (DM), and where relevant a BCT was assigned. These were subsequently reviewed by SW. We did this to provide a reliable record of the content of this behaviour change intervention, and to confirm that we included a range of BCTs as planned.

Results

This results section describes the key steps in our website development.

Phase 1: initial planning stages

Work package 1—understanding the evidence & incorporating theory (scoping review and expert panel)

The planning stage focussed on the 5 key tasks outlined by Campbell et al. [5]. This process helped us understand that the main problems we aimed to address centred round: 1) the suboptimal use of preventative therapies; 2) the high levels of symptom burden; and 3) the low rates of attendance at asthma reviews and use of asthma action plans. We anticipated that concepts derived from behaviour change theory should help us address these identified problems, and the full results of WP 1 are described in Table 1.

Table 1 Defining and understanding the problem

Work package 2—getting user perspectives on a web resource (focus groups)

We shared our results from Table 1 with potential end users in the focus groups. Excluding the practice nurses the average age of participants was 41 years (range 23 to 56). Six participants were female, 4 male, and included participants from highest and lowest deprivation deciles (median 4, IQR 1, 8). Table 2 describes the participants, illustrating which focus group (or think aloud study) they participated in. Participants were recruited from Asthma UK volunteers (n = 5), primary care (n = 3) and hospital asthma clinic (n = 2). Barriers to optimum self-management identified by focus groups included not accepting diagnosis, difficulties keeping track of medications and remembering to order more, and the length of time between asthma reviews resulting in knowledge loss. Facilitators to using an online resource included staggering of information, a resource to bridge the gap between annual reviews and reinforcement of material covered, provision of email reminders i.e. ordering medication and flu vaccinations, resource being promoted during annual reviews and making users aware of different types of inhalers available and importance of finding one that suits.

Table 2 Demographics of participants in focus groups and think aloud studies

These barriers and facilitators were combined with those from the literature (including asthma guidelines) to provide a list of suggested features to include in a resource. This process is shown in full Additional file 1, illustrating the rationale for the contents of the website.

Work package 3—developing a draft version of the website (expert panel)

By the end of WP 1 and 2 we established there were 6 main behaviours we wanted to promote within the website:

• Recognise symptoms, don’t put up with them (aim for no symptoms)

• Optimise medication use (including inhaler technique)

• Attend for regular asthma review

• Use asthma action plans

• Increase physical activity

• Stop smoking

The expert panel reviewed the list of suggested features from Additional file 1 which led to the removal of four: a diary for tracking medication use, a diary for tracking peak expiratory flow (PEF) rate, a tailored action plan and a dedicated family & friends sections. The expert panel felt that evidence and personal experience suggested that use of diary tools was rarely sustained except by a few very motivated individuals. Instead regular prompts to think about current asthma symptoms based on the ‘Royal College Physicians 3 Questions’ (RCP 3Q) screening tool [27] was incorporated throughout the resource and in the automated emails. This asks the user about difficulty sleeping because of asthma, asthma symptoms during the day, and interference with usual activities. If users answer yes to even one question then further assessment of asthma control is indicated [28, 35]. Action plans work best when personalised to the individual [6] and the IT requirements of a truly tailored action plan was considered beyond the scope of this project. Instead a section was dedicated to promoting the use of action plans, and encouraging individuals to visit their health professional to agree one if they didn’t have one. Rather than a dedicated family and friends section the importance of positively involving family and friends was discussed in general terms.

By the end of Phase 1 we had developed paper based versions of the web pages ready for consideration by the expert panel and for use in think aloud studies.

Phase 2: iterative refinement of the resource contents of the website (think aloud studies and expert panel)

Eleven think aloud studies (see Table 2 for participant details) were conducted although one study (TA 08) was not completed as the website was not compatible with her type of computer which converted website text into braille (BrailleNote). Four of the 11 studies were undertaken in the participants’ own home. Three of the participants (participants 3, 4 and 10) undertook 2 studies each. Each think aloud interview covered a slightly different range of topics as the resource was developed iteratively (Table 3). Table 4 explains the nature of the changes made during this phase as a result of input from the think aloud participants and the expert panel.

Table 3 Think aloud studies—topics covered

Table 4 Changes made during phase 2

Thematic analysis of the think aloud transcripts identified three main thematic categories: 1) ‘content’—the actual words on the pages, and how relevant and understandable the information was; 2) ‘layout and navigation’—the layout of pages or sections, and how easy it was to navigate around sections; and 3)‘user experience’.

NVivo software allowed us to generate quantitative data from the think aloud transcripts. Fifty one percent of the comments were positive, 15 % negative and 34 % containing suggestions for improvement. This suggests that participants felt comfortable criticising or making suggestions for improvement of the website. Most comments related to the content of pages (78 %), and the majority of these were positive (56 %). In contrast, most comments about the website layout and navigation were negative (69 %) (Fig. 2). This confirmed that the ground work done in Phase 1 around content had been successful, but that greater emphasis was needed on usability and presentation issues.

Fig. 2

Type of comment made during think aloud studies

Content—making the website relevant and understandable

Participants were positive about the contents, and in particular the ‘level’ it was aimed at:

“it’s very clear in its intention, a website to help you stay healthy and manage your asthma better that’s exactly what level I’m at, I don’t have a detailed knowledge of what I’ve got or quite what I’ve got or quite how to look after it so it’s perfect for me.” (Participant 10,TA 01)

Users liked and identified with the key messages, for example that people with asthma should be ‘aiming for no symptoms’:

“I like a message of you know that’s what you should be aiming for, it might not be what you get right enough but at least you should be aiming for, or aiming for it the majority of the time, you know but you can if, you know going to have relapses, but I think that that’s really good because I don’t think many people actually say that to you to be honest.” (Participant 1, TA02)

“That’s good to know because again I just was putting up with it like if I was, if I wasn’t being able to breathe I would just be like oh I'm just having a bad day rather than being like ‘oh I should really be on the brown inhaler to stop this from happening’,” (Participant 3, TA04)

While there was universal agreement that quotes from patients and practice nurses were desirable within the website there was some disagreement about how they should be presented:

“But I would give them maybe slightly more weight if they weren’t anonymous bizarrely. And it’s a real living patient that is living with asthma. And that kind of makes it more of a human.” (Participant 10, TA01)

In the following think aloud study this point was brought up by the interviewer:

“the quotes do you think, would you prefer to see something like female age 53 or is it not relevant? (researcher)

It’s not relevant to be honest because if I was twenty one and I was reading and they were fifty I would be thinking oh that doesn’t apply to me yet. The guy will be reading it and thinking oh that’s a woman thing.” (Participant 1, TA02)

Consequently, we kept quotes in the website but removed descriptions of who said them, as illustrated in Fig. 3.

Fig. 3

Screenshot of webpage illustrating use of quotes

While patients on the whole agreed with the information provided, the one area where there was scepticism was in regard to how approachable participants’ practices nurses were:

“just trying to imagine sort of sitting down with my asthma nurse and saying I have a goal and this is what I want to achieve, I know what she’d say, she’d say I haven’t got time to discuss this! Let’s just stick to the tick boxes shall we?“(Participant 4, TA05)

Layout and navigation—making the website easy to use

The majority of the comments regarding layout were page specific such as feeling that a given paragraph was too long, and where appropriate were acted on immediately after the think aloud study in preparation for the next one. However the importance of getting the home page right was clearly important to participants and generated discussion.

“it doesn’t quite feel like a home page, that’s maybe not helpful. I’m trying to think what the best way to, it looks the same as every other page, I don’t know if you did something different to the header or something like that.” (Participant 4, TA05)

Therefore the home page was modified as illustrated in Fig. 4 in response to comments across the studies.

Fig. 4

Changes made to home page during Phase 2

The second recurring theme related to users ‘knowing where I am’.

“I say I might have said before maybe a little site map you are on step 3 of 9, 4 of 9 and people know where they are going.” (Participant 4, TA09)

As a result it became more obvious which section a user was in at a given time, and within the 4 week challenge section it was made much more obvious how users progressed through the 4 stages of preparing to sign up to the ‘4 week challenge’.

User experiences

After completing the think aloud study users were asked how they might use the website in a real life setting and what would be barriers to its sustained use. Users felt that they would have more confidence in such a resource if a health professional recommended it:

“I guess like in my annual review, if my nurse was like oh have a look at this. Like a wee leaflet or a wee business card or something like that and just was like have a look at that.” (Participant 2, TA03)

This finding is relevant for both future large scale RCTs, and the subsequent implementation and embedding of such a resource.

Completion of this phase resulted in the final website ready for evaluation in the RAISIN trial [23]. Table 5 describes the final contents of the resource, and further sample screenshots are provided in Additional file 2.

Table 5 Contents of Living Well with Asthma resource

BCTs present in website

We incorporated 20 BCTs in our Living well with Asthma website as described in Table 6. The most commonly used BCTs were ‘information about health consequences’ and ‘demonstration of the behaviour’, followed by ‘problem solving’ and ‘instruction on how to perform a behaviour’. We also used ‘goals and planning’ as a key behavioural technique within the website.

Table 6 Behaviour change technique mapping of Living Well with Asthma resource

Discussion

We have demonstrated the feasibility of developing an evidence-based, theory guided, user friendly behaviour change intervention in the form of Living well with Asthma—a website to support self-management in adults with asthma. We have been guided by the MRC Framework on developing and evaluating complex interventions, and as a result directed much effort to the key, yet often overlooked, planning stages [5, 18]. We undertook recommended key tasks to guide our development methods [5], see Table 1, through: synthesis of empirical evidence, using expert knowledge and experience, and incorporating theoretical concepts with end user input, to produce an evidence based behaviour change website. Our evidence synthesis highlighted that self-management of asthma is an underused, yet proven, treatment strategy [7] and that people with asthma frequently do not use therapies optimally, tolerate high levels of symptom burden, and do not attend asthma reviews or make use of asthma action plans. This underestimation of symptoms and overestimation of control is a barrier to making use of available therapies. Those who do recognise they are experiencing asthma symptoms often do not adhere to therapies, often due to perceived misunderstandings around medication side-effects, or lack of perceived benefits to using asthma action plans. This analysis provided us with pathways of how a behaviour change intervention might work, focussing on behaviour change concepts recommended in the literature we developed a list of features which a website should have. This list was iteratively modified with input from end users and an expert panel, until a draft of proposed web pages had been developed. These were then gradually converted to working interactive webpages and refined over 10 think aloud studies, to lead to the final website which is being evaluated in the RAISIN trial [23]. Our BCT mapping exercise demonstrates that the resource makes use of multiple BCTs, a strategy which in some health domains has been associated with increased effect sizes [37]. In particular we use goals and planning as a key behavioural technique, which has been shown to be efficacious in asthma [1].

Strengths

This study followed recommended processes for developing complex evaluations, and was undertaken by a multidisciplinary team with a range of essential skills, knowledge and experience (including behaviour change theory and implementation theory). A key strength of this resource is in its co-design with potential end users, who had opportunity for input both at the early development planning stages in the form of focus groups, and also towards the end where their input via think aloud studies was invaluable in improving the usability of the resource, in line with the Person Based Approach [15]. The use of LifeGuide software allowed for a streamlined and iterative process of website development where the researcher taking the think aloud studies could modify the website directly following think aloud studies, or from feedback from the expert panel. Most computer programmers do not have a background in healthcare, and therefore removing the need to communicate user feedback to a programmer by using LifeGuide made the process far more efficient.

Limitations

In the focus groups we invited both practice nurses and adults with asthma which could be construed as a limitation. However there are advantages to bringing together a diverse group of participants and we felt this was the case here [40]. This can maximise the exploration of different perspectives, which was pertinent here where differences in health professional and patient opinion is a recognised barrier to optimal uptake of self-management practices [31].

The adults with asthma participating in the focus groups and think aloud studies had more severe asthma and were on more treatments than typical primary care patients. This is almost certainly because of them being recruited through their participation in asthma advocacy organisations (Asthma UK). We managed this by tempering the suggestions and feedback from these end users with the practical experience of the respiratory physicians and GPs on the expert panel and the practice nurses present in the focus groups. In future studies concentrating recruitment to end users more typical of a primary care population would be worthwhile, although may be difficult.

The same researcher who developed the website, also undertook the think aloud studies. While this had benefits in terms of speed of modifying the resource, we were concerned that participants in the think aloud studies may not have felt comfortable criticising the resource openly in the presence of the person who was also developing it. In order to counter this it was explained that it was easy to make changes with the LifeGuide software and those critical comments were often the most helpful. Exploring the scope of this limitation by counting negative comments was useful, as the high proportion of negative comments or suggestions for improvements suggests that participants did feel comfortable being critical of the website.

Future considerations

The ultimate aim of following the updated MRC guidance on the development and evaluation of complex interventions is to reduce the number of interventions which are developed, but don’t translate into everyday use, and avoiding costly large RCTs which due to unforeseen circumstances are unable to answer the research question posed [18]. The iterative methods of development used here should minimise this risk, and the ongoing RAISIN pilot RCT should allow for meaningful estimates of effect sizes and recruitment and retention rates for any future full scale randomised controlled trial. A qualitative evaluation, involving participant interviews and using NPT to guide analysis, is embedded in the RAISIN trial and will provide rich data on the how the intervention can be improved and its future implementability in the real world [24].

Conclusion

We have developed a resource which our preliminary usability testing suggests is relevant and usable by its target audience. We have outlined the key steps undertaken which included synthesis of knowledge and experience from our expert panel, with a broad exploration of the literature, overarching use of appropriate theory (behaviour change and implementation) and also with input from potential stakeholders (adults with asthma and practice nurses) from an early planning stage. Such methods are rarely fully detailed in the literature and thus the description of this process should be of interest to the growing cadre of researchers developing digital interventions. This paper demonstrates how data from a wide range of sources can directly and practically influence the contents of such a self-management website.