BMC Medical Informatics and Decision Making

Background: This study was undertaken to assess the feasibility of using videoconferencing to involve community-based surgeons in interactive, multidisciplinary oncology rounds so they may benefit from the type of community of practice that is usually only available in academic cancer centres.


Background
Optimal care of the cancer patient is increasingly predicated on multidisciplinary management. While radiation oncology services and some medical oncology services are offered at Ontario's regional cancer centres, an analysis of referral patterns for cancer surgery in Ontario identified that teaching centres (which include some cancer centres) performed less than 40% of the surgical procedures for breast cancer and colorectal cancer [1]. Therefore the community-based general surgeon plays a key role in caring for patients with cancer and ensuring that they have access to multidisciplinary consultation and treatment when necessary [2,3]. Effective communication between the different care settings is crucial to this process.
Problem-solving and interaction with colleagues is a basic component of medical practice. This need is amplified when the involvement of multiple professionals is required for optimal care. Discussions with colleagues in "communities of practice" build mutual trust and foster the exchange of not only explicit knowledge that is easily codified in documents such as guidelines, but tacit knowledge, which prompts the individual to reflect upon their practice [4][5][6]. It has been suggested that, to promote knowledge transfer, organizations need to actively nurture the development of communities of practice by legitimizing participation, negotiating their strategic content, bringing together complementary expertise, and providing support in the form of guidance and resources.
A growing body of literature describes the successful use of multi-point videoconferencing to facilitate communication between health care professionals, including oncologists and surgeons; support the care of patients; and offer continuing education across great distances [7][8][9][10][11][12][13][14][15]. Many health care groups in Canada have successfully used telemedicine [16]. The Canadian Coordinating Office for Health Technology Assessment recently prepared a review of the use of multipoint videoconferencing [17]. All programs resulted in improved communication, leading to informal learning opportunities and enhanced professional skills for primary care practitioners through interaction with practitioners at tertiary referral centres.
The Cancer Care Ontario Surgical Oncology Network conducted this study to assess the feasibility of using videoconferencing technology to bring together physicians in various care settings in a format resembling multidisciplinary hospital-based rounds or "tumour boards". The goal is to enable community based surgeons to benefit from the type of community of practice that is usually only available in academic cancer centres.

Methods
The development and pilot study of Tele-Oncology Rounds Ontario (TORO) consisted of a needs assessment, content and technology planning, implementation, and evaluation. In the absence of a "how-to" manual on videoconferencing the series was designed according to information provided by a variety of stakeholders who were consulted during the development of this program. Vide-oconference services were planned to be offered through the services of the NORTH Network, a telemedicine project based at Sunnybrook and Women's College Health Sciences Centre with satellite videoconference suites in a number of locations across central and northern Ontario. This feasibility study was part of a larger study approved by the University of Toronto Health Sciences Ethics Review Committee.
A needs assessment was conducted using both direct and indirect methods. Each NORTH Network-enabled site was contacted to confirm the name and contact information for general surgeons. Direct needs assessment involved a survey of general surgeons at these sites to identify the most convenient days of the week and times at which to offer the videoconference rounds, and interest in specific topics. Indirect needs assessment was carried out using three strategies. First, responses to a previously conducted survey of provincial surgeons were examined. In that survey surgeons were asked to specify their clinical and information needs in relation to the care of cancer patients. Second, the Royal College of Physicians and Surgeons of Canada WebDiary database was analyzed to identify the cancer-related self-directed learning projects most often undertaken by Ontario general surgeons using this online portfolio [18]. Third, open-ended unstructured telephone interviews were conducted with three cancer surgeons from different regions of Ontario. One of these interviews involved a surgeon residing in Ottawa. He was interviewed for his expertise and knowledge in the concepts of knowledge transfer, organizational learning and surgeon education. The purpose of this interview was to discuss the organization and format of the pilot study rounds and how to best evaluate them. The two remaining interviews involved a surgeon from North Western Ontario (Fort Francis) and a surgeon from North Eastern Ontario (Sault Ste Marie). As regional liaisons to the Surgical Oncology program, these individuals were selected to discuss the usefulness and format of videoconference multidisciplinary oncology rounds. As general surgeons practising in remote regions of Ontario their feedback was representative of the surgeons targeted by this feasibility study.
Based on the topics identified by the needs assessment as of interest to general surgeons, surgical, radiation and medical oncologists associated with two tertiary care cancer centres were invited to participate as discussant panelists for six videoconference rounds. They were provided with a description of the venue and tips on videoconference etiquette. Another surgical oncologist was invited to serve in the role of facilitator for each event.
With a completed schedule for six planned events, the series was accredited by the Royal College of Physicians and Surgeons of Canada through Continuing Education, Faculty of Medicine, University of Toronto as a Section 1 Accredited Group Learning Activity according to the framework of educational options for the College's Maintenance of Certification program [18].
Promotional material was created and distributed by regular mail to potential participants at the selected videoconference sites. Technical/education support staff at each site were also contacted to request internal advertising of the series.
Several weeks prior to each session the presenting surgical oncologist was asked to develop a patient scenario that involved cancer surgery, presented treatment decisionmaking challenges, and would foster multidisciplinary discussion. The multidisciplinary panel (radiation oncologist and medical oncologist) were asked to review and edit the patient scenario. One week prior to each event the patient scenario was distributed to prospective participants by fax and posted to the project web site. The scenario consisted of a description of the theoretical case, learning objectives, and several multiple choice questions.
Following a review of the medical literature and communication with researchers in departments of continuing education at six academic institutions, an evaluation form was developed to assess participant satisfaction with the format of the videoconference (image, sound, interaction), the content, and the presenter, and the extent to which participants considered the discussion useful to their practice. Statements in each of these categories were graded on a five-point Likert scale, where a score of 5 indicated strong agreement with the statement and a score of 1 expressed strong disagreement. Respondents were invited to provide written comments on whether the TORO discussion might change their practice and, if so, how; what factors or resources would enable a change in practice; and general comments.
Evaluation form responses were examined using the Statistical Package for the Social Sciences (SPSS) version 10.0 (SPSS, Chicago, IL, USA). Written comments were summarized and assessed for trends.

Indirect Needs Assessment
Telephone interviews were held with three regional representatives of the Surgical Oncology Network, including those from the North West, North East, and Eastern Ontario, who agreed that videoconferencing could facilitate the development of learning communities for improved exchange of knowledge related to cancer, particularly in remote regions of the province.
In 1998 the Surgical Oncology Network conducted a needs assessment survey of surgeons in Ontario. Half of the 612 respondents (39% response rate) agreed that access to surgical oncologists for consultation would be useful. Frequently mentioned resources requested through written comments included oncology rounds, regional meetings and continuing education (CE); and guidelines for when patients should be referred to formal cancer centres.
The online portfolio offered by the Royal College of Physicians and Surgeons of Canada was searched for instances in which physicians recorded learning related to cancer care. This was carried out in lieu of mailing a survey to surgeons to identify their perceived learning needs. The analysis identified the topics most often pursued for continuing education on cancer care, and what associated factors may have prompted a change in practice. A total of 1,203 cases were recorded between 1994 and 2001. Most records involved the category of management (52.9%) of breast (25.5%), colorectal (14.0%), gynecologic (12.7%) and prostate cancer (11.4%). Questions were most often stimulated as a result of browsing the medical literature (27.5%), in response to the management of a current patient (19.0%) or after review of the management of more than one patient (16.0%). To better identify important topics for discussion at videoconference rounds, the questions of general surgeons in Ontario were analyzed for common issues and topics. Of 71 questions posed by general surgeons in Ontario using WebDiary during the specified time period, 56 (79%) involved breast cancer or colorectal cancer.

Direct Needs Assessment
Prospective participants at video-conference enabled sites (n = 51) were surveyed to learn about specific topics of interest and preferred dates/times. The 13 (25.5%) respondents suggested a variety of topics, including breast, colorectal, lung, thyroid, pancreatic, gastrointestinal and esophageal cancer, and malignant skin lesions. The clear preference for session time was at the end of the working day.

Planning and Implementation
The Cancer Care Ontario Surgical Oncology Network offered a series of six oncology rounds to cancer surgeons by videoconference during the 2001-2002 academic year. Each one-hour session consisted of interactive discussion based on a patient scenario. Topics included breast cancer (2 sessions), colorectal cancer (2 sessions), gynecologic cancer and lung cancer (1 session each).
Staff at the NORTH Network were notified in advance of the date and time for all six sessions. They handled communication with the videoconferencing bridge service and technical support staff at each site, and attended the session at the central discussant site to coordinate connectivity. The TORO coordinator recorded and summarized the discussion at each session, later incorporating citations for relevant published trials or guidelines. The summary was distributed to the oncology panelists for review and edited according to their feedback. The discussion summary was distributed with the patient scenario for the subsequent sessions by fax and by posting to the project web site.
A median of 22 physicians participated in each session, excluding the discussant panel members and the facilitator (Table 1). A median of eight sites participated in each session, excluding the central discussant panel site.

Evaluation
The proportion of participants returning completed evaluation forms for each session was 44.8% to 67.7%. Overall, 74.6% of participants agreed or strongly agreed they were generally satisfied with the event. The majority of respondents were satisfied or very satisfied with the videoconference format, all discussants and the content of each session (Table 2). Nearly 75% of respondents agreed or strongly agreed that the session topic was relevant to their practice, and that the presenter established good rapport with the audience (85.0%), stimulated critical thinking (73.2%), and encouraged interaction (83.6%). More than half of the participants agreed that discussion provided useful insights for practice, and one quarter of respondents suggested their practice would change.
Responses were examined by session topic and by participant specialty (Table 3). Participants were satisfied with the sessions on breast, colorectal and gynecologic cancer topics, and 30% to 43% agreed or strongly agreed that discussion revealed information not accessible elsewhere.
Oncologists were more satisfied than general surgeons overall (88.3% versus 75.0%) but more general surgeons than oncologists reported that information revealed was not accessible elsewhere, discussion provided useful tips for practice, and discussion caused reflection on practice.
Further, surgeons but no oncologists indicated that information obtained at these sessions would change their current practice.
Respondents were invited to provide general comments, and describe specific changes to practice and perceived barriers to implementing changes. Written remarks were few. Respondents suggested strategies for improving the sessions, for example "more didactic teaching" and "less opinion, more facts". One respondent asked that we "encourage participants to speak loudly" and be more observant of "raised hands" of those participants wishing to add to the discussion. Others noted that videoconferencing provided an "excellent forum for CE", that it was "nice to hear how other northern centres deal with this disease", and that the session was "very interesting and enjoyable because of the interaction". Several respondents noted that they would more often consider, and refer patients for, preoperative radiation therapy and chemotherapy.

Discussion
This study investigated the feasibility of conducting multidisciplinary oncology rounds by videoconference to facilitate interaction between oncologists in the tertiary care setting and community-based general surgeons, who conduct a large proportion of cancer surgery and link patients to oncology services. A needs assessment involving direct and indirect methods clearly established that breast cancer and colorectal cancer were the topics most relevant to general surgeons. Apart from the discussant panel and facilitator, one or two oncologists participated in each session, and two gynecologists participated in the session on gynecologic cancer, so the median participation rate represents approximately 40% of the target population of general surgeons associated with the videoconference-enabled sites. For the most part, the same individuals tended to participate in each session.
Satisfaction with the videoconferencing format, content and speakers was high. Overall, discussion prompted reflection on practice and 25% of respondents suggested that their practice would change. Reflection on practice and changes in practice were more likely to occur for responding surgeons than for oncologists. Although written responses were few, several participants at different sessions noted that they would more frequently consider preoperative chemotherapy and radiation therapy as part of their management plan.
These results cannot be generalized to the larger population of general surgeons in Ontario or elsewhere due to the self-selected participation of those attending the sessions, and the small number of participants. For most sessions, only half of the participants returned evaluation forms, despite the fact that this was a requirement for receipt of continuing education credits toward the RCPSC Maintenance of Certification program.    It should be noted that this feasibility study was part of a larger study that did not take place due to an insufficient number of consenting participants. The larger study planned to conduct pre-and post-videoconference series surveys to examine self-directed learning attitudes and behaviour based upon Oddi's validated self-directed learning inventory [19]; dialogue analysis of videotaped videoconference rounds to examine instances of selfreflection, community building and informal learning according to Wenger's communities of practice framework [20]; and follow-up semi-structured, grounded interviews with participants to assess impact on behaviour and practice.
Our inability to more thoroughly assess the impact of participation in videoconference multidisciplinary oncology rounds revealed a variety of factors that challenge valid evaluation of remote collegial interaction. These same challenges were noted in the medical literature on videoconference applications. The use of telemedicine has been primarily driven by perceived need and practical attempts to deliver health care in an efficient and timely manner, rather than a theoretical knowledge base established by empirical research [21]. For this reason there is a lack of appropriate evaluation frameworks by which to assess and value the benefit of telehealth initiatives [22]. The International Network of Agencies for Health Technology Assessment (INAHTA) recently reviewed the factors that could be considered when assessing telemedicine applications [23]. They argued that controlled scientific studies of telemedicine initiatives may only be helpful in establishing the efficacy of the technology, but the results of such studies will not be widely generalizable or useful in other contexts if the application is sustainable and evolves into long-term use. Furthermore, measurement of changes in participant behaviour or patient outcomes poses major methodologic challenges because the number of subjects may be small and the power of the study low, necessitating a reliance on surrogate indicators of effectiveness. This explains why a systematic review of studies assessing telemedicine applications found that the majority were pilot projects evaluating short-term outcomes and most were of low quality [24].
The INAHTA review noted that definitive assessment of a telemedicine application may take a considerable time and be complicated by both changes to the technology and to the healthcare system, suggesting the need for a series of rapid, less detailed evaluations to provide decision makers with timely interim advice. Indeed, at the time of this writing, the Surgical Oncology program had undergone an organizational restructuring and videoconferenced oncology rounds were no longer offered. It also suggested that telemedicine applications should be evaluated according to their phase of implementation because such programs need to sufficiently mature for assessment to be meaningful. Therefore, an initial assessment associated with a pilot study, such as the one described in this manuscript, can indicate the feasibility of an initiative, while a more complex evaluation strategy, including examination of technical issues, impact on practice, and cost, is more suitable for long term assessment.

Conclusions
Ontario surgeons participating in videoconference oncology rounds reported satisfaction with the format, noting that participation stimulated reflection on practice and that, in some cases, current practice would change. Therefore ongoing assessment of the use of videoconferencing for continuing education and the development of professional communities of practice is warranted. However further research is required to develop frameworks and suitable, phase-specific methods by which to evaluate the impact of telehealth initiatives.