Table 4 Sample health care provider, administrator, and health IT interviewee quotes by topic

Characteristics of the user

Perceived utility

I don’t know necessarily that it would change the surgeons chances of offering surgery (Administrator #8)

It would be a good tool for the provider to say, “This is a tool we use and we don’t think you’re a good surgical candidate and here’s the reason why, and we’ve got numbers to show you. (Non-surgeon health care provider #1)

People want to hear percentages. I’ve always found that a very difficult question to answer (Non-surgeon health care provider #9)

There is a large portion that by the time they get to the clinic, they are expecting surgery and they want surgery. And I don’t know that they are going to necessarily care about these numbers. (Non-surgeon health care provider #1)

I would need to know a lot more about it before I would use it. (Administrator #10)

Clinical content

Quality and applicability of the calculator

I personally would be totally opposed. But if it were to be that really validated, then I don’t see a barrier. (Administrator #10)

I would say surgeons should not use a calculation that does not take into account opioid use. (Surgeon #1)

Redundancy of information

Patients checking in are already given [questionnaires], which they’re not really that happy with. So giving them another [set of questions] could be a little overwhelming. (Administrator #8)

That’s another thing with these patients, if they start seeing repetitive answers, then they just get frustrated. (Non-surgeon health care provider #4)

Workflow and Communication

Selecting who should complete the calculator

I think it may be more beneficial to do it all across the board from an operations standpoint, because it would be too hard for one person to decipher who needs the calculator and who doesn’t. (Administrator #8)

It could create a lot of confusion in the patient’s mind if they got a decent predictive number for surgery and we were not offering them surgery. Or the other side, if they were given some sort of horrible predictive value, but they came in and they needed to have surgery. That might make it a bit more confusing. (Non-surgeon health care provider #8)

Determining when to administer

I think that this should be completed in patients, where surgery is being considered an option, but before the decision to pursue surgery has been made. (Surgeon #5)

If they’re a candidate, but haven’t quite made that decision for surgery or haven’t formally been offered surgery, I think that would be that sweet spot. (Surgeon #12)

Getting providers to use it

It’s going to take some time. They have to get used to the measure. They have to get familiar with it. They also have to see how it behaves in their population. (Administrator #1)

Generally we’re in the chart a little beforehand, just to look at whatever the past note or imaging before we go into the room. So if [a reminder] is in there, then we’ll go see the patient. (Surgeon #2)

It’s got to be in their face, at faculty meetings, individual meetings, really making them pay attention to the information so they’ll use it. (Administrator#14)

Human Computer Interface

Communicating probabilities and key terms

Like the disability score of less than 22, I don’t feel like a patient is going to really know what that means. (Non-surgeon health care provider #3)

Providing an explanation for how you define disability. Is it your ability to do your activities of daily living? Is it your standing, sitting? Is it those physical movements? (Administrator #14)

Resource needs and constraints

An outcomes coordinator might be a potential resource that would be required to truly make sure that every patient that needs to fill out the survey is filling out the survey and to follow up. (Administrator #9)

It would definitely require human resources. Any time we ask a patient to either complete it via My Health, someone has to make sure that it’s done. And if it’s not complete, you then have to reach out and get that information. And any time you have human resources dedicated to working, you’re going to have a financial component to that. (Administrator #6)

Internal Organizational Policies, Procedures, and Culture

Culture of research and innovation

I have found our surgeons to be incredibly open to new in new innovations, especially ones that aren’t going to take a lot of their time. (Administrator #10)

Because we’re a teaching facility, everybody’s always on board. So that’s a positive and a negative, right? The negative is, because we’re a teaching hospital, there’s always something new, something that we are trying to figure out. So then you have those people who don’t get on board as quickly or as much as you would hope that they would. (Administrator #3)

Segmented and siloed care delivery

The compartmentalization of the different departments and service lines would be an issue. (Surgeon #8)

The same problem exists here that exists worldwide and creates all of the problems that we all deal with every day, bad communication. (Administrator #14)

External Rules, Regulations, and Pressures

Issues regarding payment

There may be the opportunity to come into agreements for more expedited approvals of surgeries, if [payers] were aware that we are utilizing this tool. (Administrator #9)

I would think if this is something that [payers] get their hands on, they’re going to look at it and say, “Well, there’s only a so-and-so percent chance that you’re going to improve. We’re not going to pay for it. (Administrator #4)

I think a calculator like this could in some instances maybe harm the chances of getting it approved. And then in some cases it might help it. Hard to say (Non-surgeon health care provider #6)

Issues regarding liability

Maybe there should be some sort of disclaimer that this is just a tool that we are using to try to help and should not be something that’s solely relied upon. (non-surgeon health care provider #8)

I’ve got to be really careful because I’m going to have people second guessing me in the way that I usually don’t use this calculator. (Surgeon #7)

System Measurement and Monitoring

How to measure success

Does [deciding on surgery] happen quicker when you bring up the calculator? Does that seem to be changing patient behavior on the quicker side to want to move forward with surgery?“ (Administrator #14)

If you’re not affecting the patient’s decision-making, then you’re useless, because, I don’t think you’re going to affect my decision-making with this. So, you got to find out if it affected their decision, whether or not to have surgery (Surgeon #15)

The easiest and most superficial way would be to ask a patient, “Did you like that? Did you like having access to that information?“ (Surgeon #6)