Skip to main content

Validity and usability of a professional association’s web-based knowledge translation portal: American Physical Therapy Association’s


Background is an evidence-based, on-line portal created by a professional membership association to promote use of evidence in practice and to help decrease unwarranted variation in practice. The site contains synthesis documents designed to promote efficient clinical reasoning. These documents were written and peer-reviewed by teams of content experts and master clinicians. The purpose of this paper is to report on the content and construct validity as well as usability of the site.


Physical therapist participants used clinical summaries (available in 3 formats--as a full summary with hyperlinks, “quick takes” with hyperlinks, and a portable two-page version) on the site to answer knowledge acquisition and clinical reasoning questions related to four patient scenarios. They also responded to questions about ease of use related to website navigation and about format and completeness of information using a 1–5 Likert scale. Responses were coded to reflect how participants used the site and then were summarized descriptively. Preferences for clinical summary format were analyzed using an analysis of variance (ANOVA) and a Dunnett T3 post hoc analysis.


Seventeen participants completed the study. Clinical relevance and completeness ratings by experienced clinicians, which were used as the measure of content validity, ranged from 3.1 to 4.6 on a 5 point scale. Construct validity based on the information on the site was supported for knowledge acquisition questions 66 % of the time and for clinical reasoning questions 40 % of the time. Usability ratings for the full clinical summary were 4.6 (1.2); for the quick takes, 3.5 (.98); and for the portable clinical summary, 4.0 (.45). Participants preferred the full clinical summary over the other two formats (F = 5.908, P = 0.007). One hundred percent of the participants stated that they would recommend the PTNow site to their colleagues.


Prelimary evidence supported both content validity and construct validity of knowledge acquisition, and partially supported construct validity of clinical reasoning for the clinical summaries on the site. Usability was supported, with users preferring the full clinical summary over the other two formats. Iterative design is ongoing.

Peer Review reports


Evidence-based practice (EBP) integrates patient values, clinician expertise, and best available evidence to provide the best care [1]. Physical therapists (PTs) have acknowledged the importance of EBP in the American Physical Therapy Association (APTA) vision statement [2]. Physical therapists have a positive attitude toward using evidence [3, 4] and believe that using EBP is necessary; however, it has been difficult to implement the principles to inform examination and intervention [36].

Physical therapists and other health care professionals frequently report lack of time as the largest barrier to using evidence in practice [35]. Decreased confidence in literature search and appraisal skills is also a reported challenge [4]. In a survey of a random sample of 488 APTA members, Jette et al. [3] found that most agreed that EBP was necessary, that literature was helpful in their practices, and that the quality of patient care was better when evidence was used. However, 84 % indicated they needed to increase the use of evidence in their daily practice [3].

Knowledge translation (KT) and Knowledge to Action (KTA) frameworks provide a structure for assessing gaps in evidence-to-practice and for creating resources to fill those gaps [7]. KT strategies have been synergistically enhanced with the development of technology [8]. Evidence databases, specialized search engines, and website portals are available to help therapists’ access evidence more easily. The use of online resources has been associated with positive behavior change in health care workers including nurses, physicians, physical therapists, and occupational therapists [913].

The PTNow ( evidence-based portal was created by APTA to help reduce unwarranted variability in practice and promote the use of EBP. contains clinical summaries written and peer reviewed by clinician and scholarly experts. The clinical summaries aim to synthesize the literature in a clinically accessible manner organized by the patient/client management model from the Guide to Physical Therapist Practice [14]. There are three formats for the clinical summaries: full clinical summary, similar to a monograph, and "quick takes" (see Fig. 1) and a portable (pdf) clinical summary (see Fig 2). The clinical summary and "quick takes" are organized with tabs and hyperlinked to resources on and off the site; and the portable printable summary is a two-page precis.

Fig. 1
figure 1

Example of Quick Takes

Fig. 2
figure 2

Example of portable summary (page 1)

Validity and usability are essential elements in guiding website design. Rehabilitation-focused evidence-based online resources such as Rehabilitation Measures Database [15] and StrokEngine have provided evidence for their content validity by asking users if the content was clinically relevant [13]. Additionally, both entities assessed site usability by having users comment on how easy it was to find and use the information on the site. However, construct validity for facilitating clinical reasoning was not established. Specifically, users’ knowledge or clinical reasoning skills were not tested in previous studies of on-line knowledge translation portals. aims to be a knowledge synthesis resource that both provides content and guides clinical reasoning. A user-centered approach, querying focus groups of multiple constituents, informed the original design. Currently, iterative design and usability testing are refining the portal to ensure validity and make it useful and user friendly. Concurrently, site validity is being explored for content completeness (content validity) and interaction with the resources for gaining knowledge and making clinical decisions (construct validity). These steps are consistent with standard procedures whereby usability studies measure the effectiveness, efficiency, satisfaction, learnability, and accessibility of a new website [16]. The purpose of this study was to (1) provide evidence about content validity by users’ ratings of the resource’s clinical relevance and completeness (2) provide evidence about construct validity of participants’ knowledge acquisition and clinical reasoning by having them respond to patient scenarios using, and (3) report on participants’ perspectives on usability.



Practicing physical therapists were surveyed at two time points as part of the ongoing evaluation of the developing website. The first survey was conducted in 2012 and included questions about ease of use, knowledge acquisition, and clinical reasoning. Additional questions were added to the 2013 user study, including questions about internet use and clinical decision making in practice. Between the first survey in 2012 and the second survey in 2013, was enhanced with:

More links to outcome measures and psychometric information

Search ability using G-code categories for functional limitation reporting

New clinical summaries

Full-text clinical practice guidelines

Participants were recruited via email in the summers of 2012 and 2013. PTs who attended national PT conferences in 2011 and 2012 who provided PTNow their contact information were initially emailed. Additional participants were recruited by purposive sampling including Catherine Worthingham Fellows of the APTA (highest honor accorded to leaders in the field, who are typically consulted on association initiatives), members of the boards of directors of the Connecticut and Massachusetts state chapters of APTA, PT professional education faculty in Connecticut, and PTs certified by the American Board of Physical Therapy Specialties (ABPTS). PTs who indicated interest were sent a consent letter via email, which indicated that initiation of the survey signified consent. The letter contained links to the survey. On entering the online survey, participants were given a unique identification number. The study was approved by the Rutgers University Institutional Review Board.

Patient scenarios

Once consented, participants were provided with links to four possible patient scenarios with the companion survey: benign paroxysmal positional vertigo (BPPV), chronic obstructive pulmonary disease (COPD), Parkinson disease (PD), and total knee arthroplasty (TKA). These scenarios were selected because they were the first clinical summaries posted on the site. The scenarios were written by one of the authors (JED) and reviewed for consistency by a second author (TJM). They contained comparable information and detail. Participants were asked to complete one or more patient scenarios and to use the clinical summaries on the site to answer corresponding questions. Each scenario was designed to take between 20 and 60 min to complete.

Patient scenarios consisted of a hypothetical case with a diagnosis of BPPV, PD, COPD, or TKA, and the participant was instructed to go to the site to answer questions related to the management of the case. These scenarios contained two parts. First, the patient was briefly described, and the study participant was asked to use to indicate how they would interview and examine the patient. After completing the knowledge acquisition questions, participants were then provided with information about examination results, and they were asked a set of clinical reasoning questions about direct patient care, patient/family education, and prognosis. To find out the format of the clinical summary that participants naturally selected (full clinical summary, quick takes and portable clinical summary), a forced choice was not imposed. An example of a patient scenario is below (see Additional file 1 for all of the patient scenarios and questions):

You are working as a home health care physical therapist, and you have just been assigned the case of a 65-year-old woman with PD (diagnosed 5 years ago and now in stage II of the Hoehn and Yahr Scale) who sustained a left hip fracture from a fall. She had her hip pinned and now is having some difficulties with bed mobility, transitional movements, ambulation in the home, and elevations. Her goals are for independence with bed mobility and with ambulation at home and in the community and for improved speed of movement. To formulate your examination strategy, you will use the information that you find in and the relevant resources in the clinical summary.


Validity was evaluated based on responses to the clinical scenarios. Content validity captures whether the universe of information is represented and is specific to the content universe as defined by the researcher [17]. Content validity was operationally defined as the participant’s ratings about whether the information in the clinical summary was complete and useful in answering clinical questions. The responses were obtained exclusively from expert physical therapists who were clinical specialists (a certification based on years of experience and completion of an exam, awarded by the American Board of Physical Therapy Specialties [ABPTS]) or who had more than 5 years of clinical experience in practice.

The clinical reasoning construct was based on cognitive flexibility theory, wherein knowledge from different concepts and perspectives is re-constructed into an ensemble used to solve the current problem [18]. The “problem” in this study was the addressing of the questions associated with the clinical scenario. Construct validity was operationally defined as the response to knowledge and clinical reasoning questions based on using Knowledge acquisition included questions related to patient interview and examination techniques as part of patient care. This construct captured general information that was required to answer the patient case questions; this information could be obtained directly from the clinical summary as well as the clinician’s prior knowledge. Knowledge acquisition responses were coded as cut and paste, paraphrase, or prior knowledge. Cut-and-paste responses were identical to the content in the clinical summary, paraphrase responses were similar to the content and organization of the clinical summary, and prior knowledge responses provided information that was not contained in the clinical summary or occurred when participants indicated that they did not use the clinical summary to answer questions in the scenario.

Clinical reasoning questions were on the topics of direct patient care, patient/family education, and prognosis based on examination findings provided in the scenario. Clinical reasoning questions required a level of synthesis and interpretation and were also evaluated for accuracy [19]. Coding for clinical reasoning questions was based on: 1) Accuracy, determined by the investigator based on a priori responses, 2) Completeness, which required a response and rationale, and 3) Used the PTNow site material in the response. Responses were coded as cut and paste, paraphrase, and prior knowledge, in the same manner as described above. Coding was checked for agreement between investigators WR and JD. In the event of a discrepancy the investigators discussed the issue and arrived at a consensus.


Usability was assessed based on participants self-report. Participants rated their user experience on the site by responding to questions from the System Usability Scale (SUS) [20]. Participants were asked to rate on a five-point Likert scale (1 = completely disagree, 5 = completely agree) on the usability and content of each clinical summary format. Site usability was defined based on whether the information was (a) easy to find (navigation), (b) easy to interpret, (c) useful in answering questions, and (d) complete. In addition, open-ended questions were available for participants to discuss usability, completeness of the clinical summary, new information learned, and recommendations. Participants also responded to questions related to demographics, background, and their use of the internet.

Data analysis

The demographic and survey responses data were analyzed descriptively, including frequencies, means, and standard deviations. Content validity based on the ratings of the clinical summary being complete and useful in answering the patient case was supported if the average ratings exceeded 4 out of 5. Construct validity was considered supported if it surpassed a threshold of two thirds of the responses (66 %) used the site, to answer the questions in the patient case. An ANOVA using SPSS Version 20.0 was conducted with a Dunnett T3 post-hoc analysis to investigate the differences in usability of the three formats of the clinical summary (full, quick takes and portable). A repeated measures ANOVA was not conducted because not all participants responded to each question and the data set, therefore, had to be treated as independent. An alpha level of .05 was set for all analysis.


One hundred and seventeen email invitations were sent to physical therapists in 2012 and 2013. Seventeen participants completed the usability study (9 in 2012, 8 in 2013). Six surveys were completed for the TKA scenario; 5 for BPPV; 2 for COPD; 2 for PD; and 2 for the comparison of PD and COPD.

Respondents had a mean age of 36.9 years (SD = 8.49 y), 81.25 % were female, and 43.75 % had received a master’s degree in physical therapy. They had practiced a mean of 12.93 years (SD = 9.18). All practice areas were represented, with the majority (38 %) practicing in an outpatient setting. Fifty six percent (56 %) were ABPTS-certified specialists, and 87.5 % were members of one or more APTA specialty sections (Table 1).

Table 1 Demographics of study participants

Internet use

All participants reported using the internet for professional activities, and 12 out of 15 reported using it for patient care. Eight participants reported using smart phones daily or weekly to assist with patient care. Of the eight participants who did not use smart phones, seven plan to use one in the future. Time spent on the internet, by activity, is reported in Table 2.

Table 2 Time spent on the internet by activity

Survey questions added in 2013

In 2013, specific survey questions about how participants used the internet for patient care were added. The responses are summarized in Fig. 3. Participants reported using Google, Physical Therapy, PubMed, and Cochrane systematic reviews most frequently in clinical practice. Fifty percent reported that it was “easy” to “very easy” to find resources on the internet to guide patient care, but only 25 % thought it was easy to very easy to determine if the internet site was trustworthy. Participants believed their practice was between 10 % to 90 % evidence based. Access to trustworthy evidence was the most frequently reported barrier to EBP.

Fig. 3
figure 3

How participants report using the internet to guide patient management

Content validity: responses related to the completeness of the clinical summaries

Clinical experts (the participants with greater than five years of clinical practice experience or a specialty certification) rated content validity based on clinical relevance and site completeness. Experts rated the clinical relevance of the site by answering the question “Was the information on the site useful in answering clinical questions?” (on a scale of 1–5, with 5 being most complete) as follows, in means (standard deviations): full clinical summary, 4.5 (.82); quick takes, 3.0 (1.3); and portable summary, 3.67 (1.5). Clinical experts also rated the completeness of the clinical summary formats (on a scale of 1–5, with 5 being most complete) as follows: full clinical summary, 4.3 (0.8); quick takes, 3.1 (1.2); and portable clinical summary, 3.1 (1.3).

Construct validity: responses to knowledge acquisition questions

A total of 32 knowledge acquisition questions were answered for the COPD, PD, and TKA clinical summaries. Participants used the clinical summary 66 % of the time to answer questions by cut-and-paste or paraphrase response strategy. Four participants used prior knowledge only to answer all knowledge acquisition questions, whereas eight participants used cut-and-paste or paraphrasing (from the content on the site) to answer knowledge-based questions. Only one participant used both prior knowledge and cut-and-paste to answer clinical questions (see Table 3).

Table 3 Responses for knowledge acquisition questions

Construct validity: responses to clinical reasoning questions

A total of 42 clinical reasoning questions were answered for the COPD, PD, and TKA patient scenarios. Fourteen answers were coded as incomplete because participants provided no rationale. Twenty-seven of the 28 complete answers were correct. Of those, 40 % were based on the clinical summary (combining cut-and-paste and paraphrase answers) and 60 % used prior knowledge. To answer clinical questions, seven of the 12 participants used prior knowledge exclusively, and five of those seven had 15 years of clinical experience or more (see Table 4).

Table 4 Responses for clinical reasoning questions

Site usability for 2012 and 2013

A total of 19 usability surveys (some participants completed more than one clinical summary) were completed. Usability of the TKA clinical summary was rated six times; BPPV, five times; COPD, four times; and PD, four times.

Participants used the full clinical summary 78.9 % of the time; quick takes, 57.9 % of the time; and the portable clinical summary, 26.8 % of the time. On average, participants “agreed” to “strongly agreed” that the full clinical summary was useful (4.6) and rated it highest in usability over the portable clinical summary and the quick takes formats. There was a significant difference in usability among formats (F = 5.908 and P = 0.007). Full summary was rated as more useful in comparison to the Quick Takes (Dunnett T3 = −1.06, P = 0.002) (see Table 5).

Table 5 Differences in Usability among the 3 Clinical Summary Formatsa

Responses to open-ended questions

Based on the responses to open-ended questions, 100 % of the participants would recommend the site to a colleague. More than 50 % of participants reported that, after reading the clinical summaries, they learned new or relevant information (Fig. 4). Information that was frequently reported as “new” was related to pathology, diagnosis, medications, and outcome measures. Participants reported that the site was easy to use, helpful, and evidence- based and that they enjoyed that the information was synthesized for the clinician. Fifty percent of the open-ended comments supported the approach of the site relative to evidence-based content and organization of information (Fig. 5). One user reported, “It’s a great site if there’s a condition you are not all that familiar with or something you’d like to refresh or learn more details about. Seems like a really nice referenced resource.” Participants also suggested improvements in site usability related to formatting and navigation. For example: “Move Full Clinical Summary and References to be seen towards the top of the page. I did not see it until I scroll all the way through the other sections. Putting it towards the right of the page instead of having ads would be beneficial.”

Fig. 4
figure 4

Comments about new or relevant information

Fig. 5
figure 5

Comments about site usability and approach


In this study we aimed to provide evidence about content and construct validity of and to determine participants’ perspectives on usability. Content validity was supported. Construct validity for knowledge acquisition was also supported. Construct validity for clinical reasoning related to patient scenarios using the PTNow site was partially supported. The site was rated as usable and useful with 100 % of participants reporting that they would recommend it to a colleague. Study participants preferred the full clinical summary, rating it 4.6/5 compared with a rating of 4.0/5 for the portable clinical summary and 3.5/5 for the quick takes.

Construct validity: knowledge

Construct validity for knowledge was demonstrated based on the percent of responses that used cut-and-paste and paraphrase. The majority of participants used the clinical summaries to correctly answer knowledge acquisition questions. Further, in the open-ended responses, more than 50 % of participants reported learning something new or relevant. Evidence of knowledge acquisition is a common outcome for knowledge translation studies [2123]. However, of greater interest to us was the use of the knowledge to make a clinical decision.

Construct validity: clinical reasoning

Construct validity for clinical reasoning was only partially supported based on the questions that required clinicians to synthesize examination findings using the best available evidence and to make educated decisions about patient care [24]. Participants used prior knowledge (60 %) more frequently than paraphrasing (28 %) and cut-and-paste (12 %) to answer clinical reasoning responses. This may in part be explained by the level of experience of the clinicians (mean = 11 years) and the number (3/7) of clinical specialists. The fact that 28 % of the responses were paraphrased suggests that even experienced clinicians might have adapted their critical thinking based on interacting with We speculated that with repeated exposure and familiarity with the resources, the paraphrasing strategy might increase. We speculate that testing people with less clinical experience may yield higher scores for construct validity.


With the exception of quick takes, usability ratings were high for navigation and interpretation of information. Unexpectedly, participants preferred the full clinical summary and portable clinical summary over the quick takes. We had hypothesized that “quick takes”--which, unlike the portable clinical summary, is linked to additional resources such as tests and measures and patient education materials--might be found more useful than the portable summary. It is likely that, in the context of the study, participants chose to use a familiar monograph style. We speculate that in a clinical setting with a time limit, preferences for the 3 formats of the clinical summaries might differ from those indicated by this study. It is important to emphasize that we did not force a choice of format, but rather allowed the participant to select. This design choice provides insight into the participant’s preferences. The responses to the open-ended questions suggest that participants liked the format of the clinical summaries. They learned from the clinical summaries and found the organization logical. In the context of the current study, the usability of the clinical summaries was supported.

Our findings are comparable to previous work on knowledge translation resources designed for rehabilitation that were tested for content validity and usability. Usability testing of StrokEngine found users to be “very to extremely satisfied” with layout/organization, quality, and clinical relevance [13]. Interestingly, this group also reported that StrokEngine had significantly higher usability scores (mean = 43, SD = 4) (P < 0.005) than Cochrane Reviews Database (mean = 26, SD = 8), Royal College of Physicians (mean = 20, SD = 5), and general internet search (mean = 26, SD = 7) [13]. The Rehabilitation Measures Database usability testing found the site easy to use and that information on the site was relevant to participants [15]. Our work, however, presents novel information on construct validity that relates to clinical reasoning.

Study design considerations

The number of participants studied is relatively small (n = 17); however, in usability studies, the number of participants may be small because of the task requirements and the iterative requirements for design [25]. We believed that a sample of 17 response sets was a reasonable representation based on the number of patient scenarios, the tasks required of the participants, and our plans for future user studies. The lack of a forced choice meant that, in some instances, a clinical summary format was not used and therefore not evaluated. Further study on the clinical summary format--in particular, the usefulness of quick takes--is indicated. In addition, we had to code clinical questions as incomplete when rationale was not provided. A “talk aloud” usability study would both (1) allow directing a participant to select all aspects of the site and (2) gain insights into participants’ choices of clinical summary format and insights into their clinical reasoning.

Although this type of user study is designed to create a scenario where clinicians visit and interact with a website based on clinical behavior, this is still not a true clinical situation. Usability studies are planned to determine the clinical usability in the natural setting. This will require that the site undergo responsive design, permitting the user at the point of care to access either on a tablet or a cell-phone.


Based on the results of this study, clinicians of different specialties and from different work settings were able to use the clinical summaries to answer knowledge acquisition and clinical reasoning questions related to the scenarios, thus providing preliminary evidence on content and construct validity. The site was created with multiple formats to meet the different needs of physical therapist clinicians. The results of this usability summary validate and support the design and approach.



Physical therapy, physical therapist


American Physical therapy Association


Benign paroxysmal positional vertigo


Chronic obstructive pulmonary disease


Parkinson disease


Total knee arthroplasty


Evidence-based practice


American Board of Physical Therapy Specialists


Doctorate of Physical Therapy


Transitional Doctorate of Physical Therapy


  1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312(7023):71–2.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Vision Statement for Physical Therapy Profession. []

  3. Jette D, Bacon K, Batty C, Carlson M, Ferland A, Hemingway R, et al. Evidence-based practice: beliefs, attitudes, knowledge, and behaviors of physical therapists. Phys Ther. 2003;83:786–805.

    PubMed  Google Scholar 

  4. Salbach NM, Jaglal SB, Korner-Bitensky N, Rappolt S, Davis D. Practitioner and organizational barriers to evidence-based practice of physical therapists for people with stroke. Phys Ther. 2007;87:1284–303.

    Article  PubMed  Google Scholar 

  5. Swinkels RA, van Peppen RP, Wittink H, Custers JW, Beurskens AJ. Current use and barriers and facilitators for implementation of standardised measures in physical therapy in the Netherlands. BMC Musculoskelet Disord. 2011;12:106.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Fruth SJ, Van Veld RD, Despos CA, Martin RD, Hecker A, Sincroft EE. The influence of a topic-specific, research-based presentation on physical therapists’ beliefs and practices regarding evidence-based practice. Physiother Theory Pract. 2010;26(8):537–57.

    Article  PubMed  Google Scholar 

  7. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13–24.

    Article  PubMed  Google Scholar 

  8. Mowatt G, Grimshaw JM, Davis DA, Mazmanian PE. Getting evidence into practice: the work of the Cochrane Effective Practice and Organization of Care Group (EPOC). J Contin Educ Health Prof. 2001;21(1):55–60.

    Article  CAS  PubMed  Google Scholar 

  9. McKenna K, Bennett S, Dierselhuis Z, Hoffmann T, Tooth L, McCluskey A. Australian hoccupational therapists’ use of an online evidence-based practice database (OTSeeker). Health Info Libr J. 2005;22:205–14.

    Article  PubMed  Google Scholar 

  10. Honeybourne C, Sutton S, Ward L. Knowledge in the Palm of your hands: PDAs in the clinical setting. Health Info Libr J. 2006;23(1):51–9.

    Article  PubMed  Google Scholar 

  11. Grimshaw JM, Santesso N, Cumpston M, Mayhew A, McGowan J. Knowledge for knowledge translation: the role of the Cochrane Collaboration. J Contin Educ Health Prof. 2006;26(1):55–62.

    Article  PubMed  Google Scholar 

  12. Magrabi F, Westbrook JI, Coiera EW, Gosling AS. Clinicians’ assessments of the usefulness of online evidence to answer clinical questions. Stud Health Technol Inform. 2004;107(Pt 1):297–300.

    PubMed  Google Scholar 

  13. Menon A, Korner-Bitensky N, Chignell M, Straus S. Usability testing of two e-learning resources: methods to maximize potential for clinician use. J Rehabil Med. 2012;44(4):338–45.

    Article  PubMed  Google Scholar 

  14. Guide to Physical Therapist Practice 3.0. []

  15. Moore JL, Raad J, Ehrlich-Jones L, Heinemann AW. Development and use of a knowledge translation tool: the rehabilitation measures database. Arch Phys Med Rehabil. 2014;95(1):197–202.

    Article  PubMed  Google Scholar 

  16. Rubin J, Chasnell D. Handbook of Usability Testing. How to plan, design and conduct effective tests., 2nd edn. Indianapolis, IN: Wiley Publishing, Inc.; 2008.

    Google Scholar 

  17. Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice (3rd Edition). Prentice Hall: Upper Saddle River, NJ; 2009.

    Google Scholar 

  18. Spiro RJ, Coulson RL, Feltovich PJ, Anderson D. Cognitive flexibility theory: Advanced knowledge acquisition in ill-structured domains. In: Proceedings of the 10th Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum; 1988.

    Google Scholar 

  19. Higgs J. Fostering the acquisition of clinical reasoning skills. N Z J Physiother. 1990;18:13–7.

    Google Scholar 

  20. Brooke J. SUS: a “quick and dirty” usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland L, editors. Usability Evaluation in Industry. London: Taylor and Francis; 1996.

    Google Scholar 

  21. Menon, Korner-Bitensky N, Kastner M, McKibbon KA, Straus SE. Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med. 2009;41:1024–32.

    Article  PubMed  Google Scholar 

  22. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7(1):50.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA. 2008;300(10):1181–96.

    Article  CAS  PubMed  Google Scholar 

  24. Smith M, Higgs J, Ellis E. Effect of experience on clinical decision making by cardiorespiratory physiotherapists in acute care settings. Physiother Theory Pract. 2010;26(2):89–99.

    Article  PubMed  Google Scholar 

  25. Bastien JM. Usability testing: a review of some methodological and technical aspects of the method. Int J Med Inform. 2010;79(4):e18–23.

    Article  PubMed  Google Scholar 

Download references


Wendy Romney was partially supported by Rutgers University, School of Health Related Professions, Rehabilitation and Movement Sciences PhD Fellowship.  The American Physical Therapy Association provided in-kind services in the form of survey assistance. Publication costs were funded through the Rivers Lab by Judith Deutsch. 

Author information

Authors and Affiliations


Corresponding author

Correspondence to Judith E. Deutsch.

Additional information

Competing interests

JR works for the American Physical Therapy Association, the organization that sponsors the portal. To guard against potential conflicts of interest, she did not contribute to the data collection and analysis.

Authors’ contributions

JD, JR, and TM responsible for conception and design. JD and WR recruited participants and analyzed and interpreted the data. JD and WR drafted the manuscript. All authors revised the manuscript and read and approved the final manuscript.

Authors’ information

Judith Deutsch, PT, PhD, FAPTA is a Professor and Director of the Research in Virtual Environments and Rehabilitation Science (Rivers) Lab in the Department of Rehabilitation and Movement Sciences (RMS) and the Graduate School of Biological Sciences at Rutgers University in Newark, NJ. She serves at the co-chair of the In that capacity she oversees site development and all the research related activities, which include site usability and validation as well as baseline assessments of physical therapist’s use of EBP and particularly on-line resources.

Wendy Romney, PT, DPT, NCS is Clinical Assistant Professor at Sacred Heart University in Fairfield, CT.

She is currently a PhD student at Rutgers University, advised by Dr. Judith Deutsch.

Tara Jo Manal, PT, DPT, OCS, SCS is an Associate Professor and Director of Clinical Services and Residency Training at University of Delaware. She serves at the co-chair of the In that capacity she oversees site development and participates in research related activities.

Jan Reynolds was Director of Evidence-Based Resources, at the American Physical Therapy Association, at the time this manuscript was written.

Additional file

Additional file 1:

Patient Scenarios and Questions.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Deutsch, J.E., Romney, W., Reynolds, J. et al. Validity and usability of a professional association’s web-based knowledge translation portal: American Physical Therapy Association’s BMC Med Inform Decis Mak 15, 79 (2015).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI:


  • Evidence-based practice
  • Usability