As a result of carrying out a thematic analysis of the interviews carried out with manufacturers, four high level themes emerged relating to manufacturers' views of user involvement within the MDDD process. These are as follows: Who is the user?; Methods used; Perceived value and barriers to user involvement; The nature of user contributions. The remainder of this section presents the findings of our study according to these themes.
Who is the User?
Discussions relating to the range of individuals that are consulted during the MDDD process revealed that there was a mismatch between the users that were consulted, and those that would actually use the device in practice. Some manufacturers believed that the needs of the patient do not originate from the patient themselves, and that patients' needs are better articulated through a hierarchy of health professionals including surgeons and 'clinical champions'.
: "The need actually will probably be established by the clinical fraternity....All you have to do after that is convince the people on down the chain, from the hospital clinical researchers right down to the guy in the street who says 'It's a good idea to have one of these. That's where the need is identified. It is identified really at the top, and then it's taught down, if you follow me"
In light of this, manufacturers had a preference for seeking input from more senior health care staff over less senior staff, regardless of who would actually use the device in practice. The assumption was that senior staff members were more than capable of speaking on behalf of less senior staff, even though manufacturers acknowledged that it was likely that senior staff members would never actually use the device. This was the case even in scenarios where the patient was seen to be the main user of the device. For example, Manufacturer #2 who is involved in the development of Automated External Defibrillators (AEDs), recognised that a significant proportion of the intended users of the device included members of the public, however, they did not see it as necessary to consult members of the public, but rather consulted senior health professionals in the early stages of device design and development. Input from nurses was also considered to be less desirable than input from more senior health care staff such as surgeons. For example, Manufacturer #11 identified nurses as the main user of their product, however, did not consider it necessary to consult nurses in the design and development process. It was felt that surgeons made the decisions on behalf of nurses and patients, and therefore it was logical to consult them to identify nurse and patient needs, as is articulated below.
: "Surgeons may be there making the decision or recommending which models to buy, but it might be the nurses who are actually using the unit...the surgeon has made the decision, but he doesn't necessarily have to actually work with it, so in your case, the controllability aspect, the patient actually understanding how to operate the unit, the clinician has made the decision and has prescribed that particular device..."
Surgeons were also considered by Manufacturer #11 as having sufficient knowledge to act as representatives for home patients. The motivation for this was as a direct result of the way in which the device is introduced and promoted to the patient. Surgeons do the marketing and promoting of the product to the patients, and therefore it was seen as important that the device is primarily designed according to the surgeons' requirements. In the quote below, manufacturer #11 justifies their motivation for valuing the surgeons' opinion over the patient users.
: "but, by and large, most of our market research is done with the surgeons, not with the end users, rightly or wrongly...but how it will be marketed will be through the healthcare professional, who will have to sell it effectively to the patient, show them how to use it."
Similarly, in the event of the user group being general health professionals, the opinions of a small number of 'clinical champions' was sought by Manufacturer #8 as well as purchasing representatives from the health authority. It was deemed more important to meet the needs of the individuals who were responsible for making purchasing decisions or held most influence over them, as opposed to focusing on the needs of the individuals that would be using the devices on a day to day basis. Therefore MDDD activity often seemed to be carried out a strong focus on how effort would translate to sales. In the following quote, Manufacturer #4 is asked who would be consulted in identifying design requirements for the development of the given medical device:
: "...it's through the doctors to, its [name]'s contact with the hospitals out there, now then I doubt it would be at doctor level, it would be more the managerial level of the hospital..."
: ...so in effect you are capturing a user need in that way...
: " ...yes."
P: Denotes the participant's response, 2
I: Denotes the interviewer's questioning)
Manufacturer #1 also focuses on those making/influencing purchasing decisions, which includes management and administrative staff. Manufacturer #1 articulates this shift in focus when asked who they view as their customer:
: "Orthopaedic surgeons really. Although it is used in patients, it is the orthopaedic surgeons who decide what he will use. Sell to orthopaedic surgeons. Increasingly in the UK, we have to sell to the hospital management to justify why they should be using our bone graft substitutes as opposed to any other on the market..."
Manufacturer #8 stated that health professionals are considered to be the main user of their device, However, the opinions of purchasing representatives are the primary source used to inform device design and development, followed up by consultation with a small number of 'clinical champions' (typically high profile surgeons and well known experts in their field). When questioned on what value the user may add value to the MDDD process, although couched in a humorous reply, it was clear that the usefulness of the user was ideally directly located in relation to sales relevant information.
: "The most helpful would to give me the year three sales figures with absolute confidence. Year-1...[laughs]..."
There seems to be limited overlap between the individuals that will be using the device, and those that are consulted to inform the MDDD process. In particular, priority is given to those that hold more senior positions within the health care system. Therefore, surgeons, doctors and clinical champions were seen as more valuable sources for identifying user needs as opposed to those individual that would actually use the device on a daily basis. Furthermore, from the manufacturers' point of view, the motivation for maximising sales seems to have conflated the distinction between the customer and the user. Therefore the needs of those that make purchasing decisions or indeed have most influence over these decisions are more salient than the needs of the user. This is certainly a more complex picture than the human factors engineering approach, which puts the user's needs at the centre of the design and development process.
Given the wide range of formal methods that are available to engage with users in the MDDD process, manufacturers tended to use only a very limited range of methods to capture information from users and patients. In line with the analysis above regarding the nature of the preferred user to consult, the most typical method used to gain input from users was via informal discussions with senior health professionals. Only one out of the eleven manufacturers (Manufacturer #8) stated that they regularly used some formal methods throughout the MDDD process, such as focus groups and questionnaires when developing their airway management device. Interestingly, the initial identification of a need for this device occurred as a result of six members of the company attending a postgraduate university course, which required the use of formal methods in order to explore and identify new device ideas. It was apparent that manufacturers do not feel that they have the time or resources to engage in rigorous formal user data collection methods, instead relying on a range of strategies including gut feel, instinct, and a personal belief that they understand the market place in which they operate, in order to identify and develop new devices.
The idea of employing formal methods is once again something that is not regarded as feasible, given the amount of resources available and the fact that manufacturers believe it is necessary to move quickly in order to remain competitive with their rivals. The view is that consulting a large number of individuals is problematic in itself, as every person that is approached for feedback has a different view. Informal methods, however, are seen as offering versatile and rapid solutions, hence the belief that relying on gut feel and pressing ahead when the moment feels right is a more feasible and efficient solution. This point is articulated by Manufacturer #8 below:
: "The very fact that someone is willing to talk with you almost means that they have slightly different view. You can go on asking forever. It's that balance between have we got sufficient confidence in what we have here to move forward vs. just the generation of the information....being confident enough of its assurance. That's what you constantly face anyhow. I think that's the dilemma you always have. You can ask the users till the cows come home but you never get a new product. You ask 100,000 you get 99,999 different opinions!"
Manufacturer #3 further echoes the observation that formal methods are rarely used for medical device development, however, informal discussions and observation are seen as more versatile and fit for purpose, :
:"you're introducing a medical device to people, first question they'll ask, is "what will it do to help me?" Second question will be "how long is this going to take?" In a busy clinic that's really important...If it takes too many clicks, then people won't use it because they are busy enough as it is."
: have you been able to capture that at all?
Through our interaction with users. We haven't got a specific mechanism for capturing it."
: Do you use any formal methods for converting customer needs into product development?
There was typically the belief that there was little need to consult the actual users formally regarding a new innovation, but rather contacting what they referred to as a 'clinical champion', was sufficient to qualify the feasibility and validity of a given new innovation. For example, Manufacturer #8, responding to a question regarding how the feasibility of new device ideas are qualified responded:
#8: "Every project will have a clinical champion. They will typically be involved, sometimes they come down here to meetings. We have a list of a couple of dozen clinicians that I can pick the phone up at anytime throughout the world and say "what do you think of this?".
When asked whether there are any formal methods used within their organisation, Manufacturer #6 stated that formal methods are used within his organisation, however, they are not relevant for this particular product. Once again, formal methods, in this case example, were considered to be too bureaucratic, time consuming, and not applicable given the device and development scenario. Informal methods, such as ad hoc discussions with senior health professionals were likely to identify the majority of user design needs, and were also more appropriate.
Yes we do, but I couldn't apply that to hip replacement at this point in time. If the surgeon tells me "that this catheter is a bit too stiff, and could you make it a bit softer, if you like, or a bit more flexible, yet do exactly the same task?" we will do that - it's for everyone's benefit"
In the above example, a pragmatic approach is taken by manufacturers to make modifications to devices, adopting the belief that if a problem is encountered by one individual, then it is likely to be a problem for the majority, providing it seems to be a reasonable request. The intuition of the manufacturer plays a major part in the process of developing products that are useful to the user, and responding to their needs.
Perceived value and barriers to user involvement
There was limited evidence that direct elicitation of user views was seen by manufacturers as being of value to the MDDD process. This appeared to be particularly the case in relation to patient users. For example, Manufacturer #11, discussing their device that was purely aimed at the home patient market, did not believe that patient involvement in the MDDD process was a particularly wise expenditure of resources. This was explicitly linked to the degree of influence that they have in terms of the level of power and influence they have in the levels of uptake of the device. In response to being asked whether they would like to involve the patient more in the MDDD process, manufacturer #11 replied:
"if they are highly powered (i.e. influential in terms of clinical decision making), if they have no power then we have to ethically try to make sure that we don't harm the patient and [that] what we do is for their benefit, but appealing to them may actually be a waste of resources, so we have to make sure we don't pretend that they have a sway when they really don't, you know? "
Once again, the above statement reinforces the notion that patients are seen as being at the bottom of the hierarchy of influence, and hence investing resources and effort to find out their opinion is not considered an effective of efficient strategy. Manufacturer #2, who also saw their device as being targeted at home market, acknowledged that the patient user is becoming increasingly important, particularly as they are now selling more directly to the home patient. However, the majority of individuals involved in the MDDD process are still predominantly senior health professionals as opposed to patients, who may play a stronger role in the clinical trials phase, after the device has been designed and developed. Therefore, the role of the patient user is considered to play a passive role, more in terms of verifying the value of an already developed device, as opposed to being involved in the initial design and development stages. This is discussed in more detail in the next section.
Another factor discouraging manufacturers from involving patient users in the MDDD process is the prospect of having to obtain ethical approval in order to carry out the research. When asked whether there are any difficulties in obtaining ethical approval for involving patient users in the MDDD process, Manufacturer #3 stated:
: "Yes. A huge problem. In my experience if you are not affecting patient care, patient throughput, you can get 'ethics', but if you are you won't get it, or you might but it will take years. So you have to design your trial so as not to affect patient care."
Manufacturer #1 also comments on the ethical approval process and the R&D committees that must be attended to when employing more formal methods users, both patient users and professional users in the collection of clinical data.
: R&D committees. That's another thing. You start a study and surgeon wants to collect clinical data to be sure that they want to use this. Their experience that this is the product they want to use. You have to notify the hospital R&D and then suddenly they see it as an R&D project, so they throw on their overheads and it's an extra hurdle to go through. They start reviewing it in addition to ethical board. This is what has changed in the last few years. In those days you notified the larger University hospitals. Increasingly has to be approved. Some are straightforward; others ask extra questions that delays process - UK research.
This perceived difficulty with obtaining ethical approval may be one reason why manufacturers tend not to use formal methods, or engage in systematic research activity in order to inform the MDDD process, particularly at the early stages of development. Indeed manufacturers appear to actively avoid involving professional and patient users in the process, in fear that the MDDD process could be delayed for years as a result of the ethical approval process. As identified in section 3.2, informal discussion with clinical staff is seen as a more realistic, pragmatic, and feasible route to informing the MDDD process, which can be carried out informally and hence without ethical clearance. Manufacturer #3 goes on to describe the strategy that they use to get around the challenge of obtaining ethical approval:
#3: "So you take the least hard route, to not affect patient care in any way whatsoever, and design your study to sit on the back of that. So the study might not be optimal, but at least you can do it."
Setting up device design and development activity so as to avoid the need for user involvement seems to be the 'method' of choice for some device manufacturers. Despite this being seen as a potentially sub-optimal approach, perhaps less effective in evaluating the extent to which design innovations may be accepted by users, it seems a route worth taking given the alternative of incurring long delays as a result of the ethical approval process.
The nature of user contributions
The measures used to evaluate the effectiveness of newly developed devices towards the end of the MDDD process echo the findings presented earlier, that formal methods employed in any part of the MDDD process are seen by manufacturers as having the potential of slowing the process down, and incurring additional and unnecessary costs and overheads that otherwise could be avoided if a less formal approach was taken. Most commonly, manufacturers reported that the success of a new device is typically measured by the absence of receiving customer complaints or 'bad news' emerging as a result of the device being used in the field. Manufacturer #1, in response to being asked how the success of their device is measured responded as follows:
: "The absence of bad news. The fact that we have access to the surgeons who are using the product and any adverse effects would be reported and we would be aware early on."
With regards to seeking patient feedback about the success of a medical device in providing an effective health intervention, Manufacturer #3 takes a similar 'no bad news is good news' approach to the design and functioning of their product. When asked whether any patient feedback is sought about the device, the reply was:
#3: "Not explicitly, we haven't gone out to get it, but we get feedback though the users (clinicians). It's non-invasive, so as far as the patients concerned it's not a problem to use it. No-one has said they don't want it done or had any problems with that."
Manufacturer #9, commenting on whether any formal methods are used for converting user needs into device design requirements, also indicated that they adopt a reactive stance to customer suggestions and complaints, which is their default position on such matters.
: "Erm, we certainly have regular meetings here where we will look at customer feedback, you know, whether it's customer complaints or customer suggestions or whatever, yes so, yes we do have a means to do that..."
Some evidence of formal user involvement did emerge from the interview data, however, it indicated that the majority of formal user involvement took place at the clinical trials stage, i.e. after the product had been developed, and manufacturers were at the stage of demonstrating its efficacy and clinically effectiveness. Therefore, in effect any considerations in terms of design preference of the medical device that may benefit the user in terms of their treatment or use of the device had already been made prior to this formal user involvement. Manufacturer #2 highlights the notion that the patient user is primarily seen as being of value when attempting to demonstrate the clinical effectiveness of a new medical device.. Interestingly, patient users are not seen as primarily informing the general design of the device at earlier stages of the MDDD process, at least not in a formal capacity. When asked whether formal user methods are employed within the MDDD process, Manufacturer #2 responded:
: "There is the formal method of gaining patient data which means at the moment for example we have conducted clinical trials, clinical investigations. All sorts of clinical information gathering is going on at the [hospital name]. We are also... we tend to gather information first of all on the effectiveness or efficacy of it...Any advance, any variation that is likely to happen to the unit, any proposals for change that are going to improve the machine are all tested out in the clinical environment."
Manufacturer #5 reported that the key driver for collecting formal user data, relating to the performance of a device, comes from the possibility that an organisation such as the National Institute for Clinical Excellence (NICE) may decide to investigate the efficacy of their device at some unknown point. Therefore, the key driver for formally collecting user generated data is to fulfil the potential future requirements of external standards or purchasing agencies. The motivation to collect user facing data does not appear to be borne out of an inherent belief, on the manufacturer's behalf, that this user data would add any significant value in terms of fulfilling their own need to develop more effective devices or indeed learn more about the effectiveness and efficiency of their own device.
: I think, if you take an organisation like NICE as a customer, or any of the other health technology organisations, one doesn't know whether your particular intervention is going to be assessed by them, and if so at what point, and one can see that as [the product] gains momentum and the NHS starts to look at what its spending on [this] surgery, then it may be something that NICE feel, or NICE get directed to take a look at. So it's a kind of a problem to us to understand when that's going to happen and it will certainly be a challenge; we have to therefore make sure that we are gathering the evidence in case they do."