COMMENT ON THE ARTICLE “SIXTEEN YEARS OF ICPC USE IN NORWEGIAN PRIMARY CARE” BY BOTSIS ET AL.
Marianne Rosendal, Woncas International Classification Committee and The Research Unit for General Practice, Aarhus University, Denmark
11 June 2010
We welcome the publication of a paper on the validity of the International Classification of Primary Care (ICPC)1 but have major concerns about the methods employed and conclusions drawn in this article. We urgently request the authors to consider our comments below and reinterpret their results.
The ICPC is a classification designed for primary care worldwide,2 and therefore has to meet 2 primary requirements. First, it must accommodate episodes of care from their starting-points, which often consist of non-specific symptoms or complaints that patients present to their clinicians, as well as specific clinical diagnoses (end-point classification). This makes ICPC different from secondary care classification systems. Second, it must also be applicable in developing countries where there is limited access to the sophisticated electronic medical records now common in Europe and elsewhere; it must be usable as a simple, mnemonic “paper-and-pencil” version. ICPC was rigorously developed and field-tested so that its relatively small number of specific codes would have high validity and the level of granularity that made the most clinical sense for primary care. Most of the ICPC rubrics have been based on empirical data on prevalence from international field trials, and in general, any disorder with a prevalence of more than 1 per thousand in a primary care population has a specific code. The combination of relatively coarse granularity and broad definition (for example, the single rubric of T90 for type 2 diabetes) allows for simple labelling of conditions while ensuring more accurate data aggregation and retrieval. In contrast, highly granular classifications or terminologies require hierarchical structuring or aggregation algorithms to enable data aggregation and retrieval without excluding valid cases.
ICPC is not sufficiently fine grained for use as a stand-alone classification in the current generation of electronic patient records, but its prevalence-based granularity and comprehensive coverage makes it very useful in providing the structure for data aggregation and retrieval when combined with fine grained interface terminologies.3 This combination can take 2 basic forms. The first is an extended alphabetical index with search terms (in effect, a controlled thesaurus of terms), as implemented in Norway and Denmark. Such an index may facilitate the user’s finding of codes and at the same time increase the validity of coding. The second is a formal mapping between ICPC and a more fine grained classification such as ICD or a reference terminology such as SNOMED-CT. In for example the Netherlands, formal mapping between ICPC-2 and ICD-10 has been completed, and an expert working group of the Wonca International Classification Committee (WICC) has begun work on mapping a primary care subset of SNOMED-CT to ICPC-2.4 Both approaches are intended to provide users with a wider range of diagnoses and enable more specific coding where needed – without any loss of validity in the ICPC-coding. Some electronic patient record applications also allow the user to add modifiers to the diagnostic text (e.g. localisation, time etc.).
With this in mind, we turn to the publication of Botsis et al, a retrospective study of clinical diagnoses coded and collected over a 16-year period in 12 primary care sites in Norway. These diagnoses were extracted from electronic medical records that employed versions of a clinical interface terminology, which appeared as a “pick list”, eventually including 6390 phrases representing “synonyms, specifications, and extensions of the original diagnoses that are included in the English ICPC-2 version, as well as other special terms that physicians used frequently to cover their clinical needs”. It is not clear whether these last terms were mapped to ICPC. According to the authors, clinicians could select one of the supplied terms, modify a term, or create one of their own. The software would then select the presumed “best match” ICPC code for the diagnosis.
From this description, it is quite clear that the study evaluates the interface terminology incorporated into the software, rather than the ICPC classification itself. The electronic record system linked to ICPC codes only indirectly, and it is not clear whether all of the 6390 clinical terms were mapped to an ICPC code, how maps were validated, how they were updated through revisions, or whether clinicians took any notice of which ICPC codes were assigned to the clinical diagnosis for a particular encounter. In fact, physicians were not using ICPC, they were using the Norwegian clinical interface terminology. The ICPC codes were assigned by software as a secondary procedure, with no quality control or validation procedure in place.
Given this methodology, it is not surprising that the authors found missing and “inappropriate” ICPC codes. Missing codes were present in 6.2% of cases, but only in the first version of the software. Incorrect codes were observed in 4.0% of cases, with “appropriate” coding rates of 88.9-99.7 for four common conditions in primary care (pneumonia, diabetes, tonsillitis and anaemia). The authors also describe a high proportion of “mismatches”, which appear to include all instances in which physicians added a word or words to the clinical term selected for the diagnosis. The authors made no attempt to determine whether these mismatches were meaningful or trivial. In fact, the “appropriate” rates are higher when compared to the ICD-coding in secondary care,5;6 and we find it extremely difficult to interpret the impact of the mismatches between applied GP text and the base ICPC code in the absence of any context. Are these mismatches substantial – or irrelevant? Could they be overcome by mapping to a finer grained classification? Could they be solved by an extension of the alphabetical index in the clinical terminology? Regardless, the “mismatch” is a consequence of how clinicians used the clinical terminology in the PROMED software application, and not related of the ICPC classification. The authors state that Norwegian GPs would use “invalid” ICPC-2 diagnoses. From our understanding of their methods, GPs could only change the text – not the codes themselves. This is in accordance with the high rate of correct use of code numbers.
There were also a few specific misstatements in the article: • The authors state that mononucleosis is not specified in the ICPC-2. It is coded as a general infection in the A-chapter as A75. • The authors state that ICPC-2 has 25 entries for glucose-related problems. This further illustrates their general confusion between ICPC classification and interface terminology, as it is the Norwegian clinical interface terminology that contains the 25 entries for glucose-related problems. ICPC-2 includes two rubrics for diabetes: T89 (Diabetes, insulin dependent) and T90 (Diabetes, non-insulin dependent). It is possible to also use: T29 (Endocrine/metab/nutrit/sympt/complt other), T99 (Endocrine/metab/nutrit dis other), a code to describe an abnormal glucose test (A91, Abnormal result investigation NOS) or a code from the process section of the classification, or a patient concern about possible diabetes. Furthermore, ICPC includes: A94 Neonatal diabetes, F83 diabetic retinopathy and W85 Gestational Diabetes.
We agree with the authors’ point that “clinical practice requires accurate diagnoses that reflect the patients’ clinical problems”. We also agree that the symptom and diagnostic sections of ICPC-2, which contain fewer than 1000 classes, do not capture the complexity of individual patients’ clinical problems. That is why we have created and published maps linking ICPC-2 to ICD-10, and why we are working on mapping SNOMED-CT to ICPC-2 for use in electronic patient records. The combination of a robust primary care classification that provides the data structure to facilitate data aggregation and retrieval (ICPC-2) and a more fine grained standard clinical terminology mapped to the classification meets clinicians’ needs for specificity as well as epidemiologists’ needs for accurate aggregation.
However, we strongly disagree with the authors’ conclusions about the usability of ICPC in primary care, as it is based on a fundamental misunderstanding about the ICPC classification. The authors evaluated a particular clinical terminology that is related to ICPC, not the classification itself. The problems identified in this study were those of the terminology and how it was implemented. This misunderstanding requires correction.
Furthermore, in reaching their conclusions the authors ignore the usefulness of ICPC in capturing and coding primary care content (reasons for encounter/symptoms and complaints, social problems affecting health) that is not included in other classifications or terminologies, instead focusing exclusively on diagnostic content. ICPC was designed to capture physical, psychological, and social problems in accordance with the bio-psycho-social model to more completely describe patients’ problems. Again, it is the structure of ICPC – encompassing the full process of care over time – that enhances its value.
Finally, we strongly disagree with the authors’ assertion that revisions underway in ICD, ICPC and SNOMED-CT “indicate either problems of structure or problems of content”. The primary challenge we face in improving healthcare information technology is finding the proper balance between usability and comprehensiveness. Each of these classifications/terminologies is evolving, but we can now see how they can be integrated: ICPC to provide structure and facilitate data aggregation, ICD and SNOMED-CT to provide fine grained classification and terminology. But in practical terms, there will be a continuing need for good clinical interface terminologies at the user interface. GPs tend to spend as little time as possible on diagnostic coding, and interface terminologies are at this point a necessary compromise. The actual search methodology that is used also has a strong effect on the usability and accuracy of the coding. However, the interface terminology has to be large enough to provide a high specificity. The authors imply that the interface terminology used in Norway is a large terminology but at 6390 terms it is relatively limited compared to the interface terminologies used in e.g. the Netherlands (94,630 search texts). It is not clear whether the problems seen in this study were the result of the relatively small size of the interface terminology or the tendency of GPs to choose their own language for diagnosis, but they were clearly not caused by the structure of ICPC.
Marianne Rosendal, Laurent Letrilliart, Kees van Boven, Anders Grimsmo, Helena Britt, Jean Karl Soler, Marc Verbeke, Marc Jamoulle, Ray Simkus, and Mike Klinkman (WICC chair)
Reference List
(1) Botsis T, Bassoe CF, Hartvigsen G. Sixteen years of ICPC use in Norwegian primary care: looking through the facts. BMC Med Inform Decis Mak 2010; 10:11. (2) Soler JK, Okkes I, Wood M, Lamberts H. The coming of age of ICPC: celebrating the 21st birthday of the International Classification of Primary Care. Fam Pract 2008; 25(4):312-317. (3) Letrilliart L., Bacis A.K., Mennerat F., et al. Interface Terminologies: A Case Study on the International Classification of Primary Care. World Academy of Science, Engineering and Technology 2009; 54:614-617. (4) Loh A., Zelmer J. Wonca and IHTSDO Seek to Strengthen Informatics in Primary Care. Launch of Cooperative Effort to Better Serve Family Doctors and their Patients. www ihtsdo org/fileadmin/user_upload/Docs_01/Press_Releases/Wonca_IHTSDO_agreement_announcement pdf [ 2010 (5) Hsia DC, Ahern CA, Ritchie BP, Moscoe LM, Krushat WM. Medicare reimbursement accuracy under the prospective payment system, 1985 to 1988. JAMA 1992; 268(7):896-899. (6) Nickelsen TN. [Data validity and coverage in the Danish National Health Registry. A literature review]. Ugeskr Laeger 2001; 164(1):33-37.
Competing interests
The authors are members of the Wonca International Classification Committee, which develops the ICPC classification.
COMMENT ON THE ARTICLE “SIXTEEN YEARS OF ICPC USE IN NORWEGIAN PRIMARY CARE” BY BOTSIS ET AL.
11 June 2010
We welcome the publication of a paper on the validity of the International Classification of Primary Care (ICPC)1 but have major concerns about the methods employed and conclusions drawn in this article. We urgently request the authors to consider our comments below and reinterpret their results.
The ICPC is a classification designed for primary care worldwide,2 and therefore has to meet 2 primary requirements. First, it must accommodate episodes of care from their starting-points, which often consist of non-specific symptoms or complaints that patients present to their clinicians, as well as specific clinical diagnoses (end-point classification). This makes ICPC different from secondary care classification systems. Second, it must also be applicable in developing countries where there is limited access to the sophisticated electronic medical records now common in Europe and elsewhere; it must be usable as a simple, mnemonic “paper-and-pencil” version. ICPC was rigorously developed and field-tested so that its relatively small number of specific codes would have high validity and the level of granularity that made the most clinical sense for primary care. Most of the ICPC rubrics have been based on empirical data on prevalence from international field trials, and in general, any disorder with a prevalence of more than 1 per thousand in a primary care population has a specific code. The combination of relatively coarse granularity and broad definition (for example, the single rubric of T90 for type 2 diabetes) allows for simple labelling of conditions while ensuring more accurate data aggregation and retrieval. In contrast, highly granular classifications or terminologies require hierarchical structuring or aggregation algorithms to enable data aggregation and retrieval without excluding valid cases.
ICPC is not sufficiently fine grained for use as a stand-alone classification in the current generation of electronic patient records, but its prevalence-based granularity and comprehensive coverage makes it very useful in providing the structure for data aggregation and retrieval when combined with fine grained interface terminologies.3 This combination can take 2 basic forms. The first is an extended alphabetical index with search terms (in effect, a controlled thesaurus of terms), as implemented in Norway and Denmark. Such an index may facilitate the user’s finding of codes and at the same time increase the validity of coding. The second is a formal mapping between ICPC and a more fine grained classification such as ICD or a reference terminology such as SNOMED-CT. In for example the Netherlands, formal mapping between ICPC-2 and ICD-10 has been completed, and an expert working group of the Wonca International Classification Committee (WICC) has begun work on mapping a primary care subset of SNOMED-CT to ICPC-2.4 Both approaches are intended to provide users with a wider range of diagnoses and enable more specific coding where needed – without any loss of validity in the ICPC-coding. Some electronic patient record applications also allow the user to add modifiers to the diagnostic text (e.g. localisation, time etc.).
With this in mind, we turn to the publication of Botsis et al, a retrospective study of clinical diagnoses coded and collected over a 16-year period in 12 primary care sites in Norway. These diagnoses were extracted from electronic medical records that employed versions of a clinical interface terminology, which appeared as a “pick list”, eventually including 6390 phrases representing “synonyms, specifications, and extensions of the original diagnoses that are included in the English ICPC-2 version, as well as other special terms that physicians used frequently to cover their clinical needs”. It is not clear whether these last terms were mapped to ICPC. According to the authors, clinicians could select one of the supplied terms, modify a term, or create one of their own. The software would then select the presumed “best match” ICPC code for the diagnosis.
From this description, it is quite clear that the study evaluates the interface terminology incorporated into the software, rather than the ICPC classification itself. The electronic record system linked to ICPC codes only indirectly, and it is not clear whether all of the 6390 clinical terms were mapped to an ICPC code, how maps were validated, how they were updated through revisions, or whether clinicians took any notice of which ICPC codes were assigned to the clinical diagnosis for a particular encounter. In fact, physicians were not using ICPC, they were using the Norwegian clinical interface terminology. The ICPC codes were assigned by software as a secondary procedure, with no quality control or validation procedure in place.
Given this methodology, it is not surprising that the authors found missing and “inappropriate” ICPC codes. Missing codes were present in 6.2% of cases, but only in the first version of the software. Incorrect codes were observed in 4.0% of cases, with “appropriate” coding rates of 88.9-99.7 for four common conditions in primary care (pneumonia, diabetes, tonsillitis and anaemia). The authors also describe a high proportion of “mismatches”, which appear to include all instances in which physicians added a word or words to the clinical term selected for the diagnosis. The authors made no attempt to determine whether these mismatches were meaningful or trivial. In fact, the “appropriate” rates are higher when compared to the ICD-coding in secondary care,5;6 and we find it extremely difficult to interpret the impact of the mismatches between applied GP text and the base ICPC code in the absence of any context. Are these mismatches substantial – or irrelevant? Could they be overcome by mapping to a finer grained classification? Could they be solved by an extension of the alphabetical index in the clinical terminology? Regardless, the “mismatch” is a consequence of how clinicians used the clinical terminology in the PROMED software application, and not related of the ICPC classification. The authors state that Norwegian GPs would use “invalid” ICPC-2 diagnoses. From our understanding of their methods, GPs could only change the text – not the codes themselves. This is in accordance with the high rate of correct use of code numbers.
There were also a few specific misstatements in the article:
• The authors state that mononucleosis is not specified in the ICPC-2. It is coded as a general infection in the A-chapter as A75.
• The authors state that ICPC-2 has 25 entries for glucose-related problems. This further illustrates their general confusion between ICPC classification and interface terminology, as it is the Norwegian clinical interface terminology that contains the 25 entries for glucose-related problems. ICPC-2 includes two rubrics for diabetes: T89 (Diabetes, insulin dependent) and T90 (Diabetes, non-insulin dependent). It is possible to also use: T29 (Endocrine/metab/nutrit/sympt/complt other), T99 (Endocrine/metab/nutrit dis other), a code to describe an abnormal glucose test (A91, Abnormal result investigation NOS) or a code from the process section of the classification, or a patient concern about possible diabetes. Furthermore, ICPC includes: A94 Neonatal diabetes, F83 diabetic retinopathy and W85 Gestational Diabetes.
We agree with the authors’ point that “clinical practice requires accurate diagnoses that reflect the patients’ clinical problems”. We also agree that the symptom and diagnostic sections of ICPC-2, which contain fewer than 1000 classes, do not capture the complexity of individual patients’ clinical problems. That is why we have created and published maps linking ICPC-2 to ICD-10, and why we are working on mapping SNOMED-CT to ICPC-2 for use in electronic patient records. The combination of a robust primary care classification that provides the data structure to facilitate data aggregation and retrieval (ICPC-2) and a more fine grained standard clinical terminology mapped to the classification meets clinicians’ needs for specificity as well as epidemiologists’ needs for accurate aggregation.
However, we strongly disagree with the authors’ conclusions about the usability of ICPC in primary care, as it is based on a fundamental misunderstanding about the ICPC classification. The authors evaluated a particular clinical terminology that is related to ICPC, not the classification itself. The problems identified in this study were those of the terminology and how it was implemented. This misunderstanding requires correction.
Furthermore, in reaching their conclusions the authors ignore the usefulness of ICPC in capturing and coding primary care content (reasons for encounter/symptoms and complaints, social problems affecting health) that is not included in other classifications or terminologies, instead focusing exclusively on diagnostic content. ICPC was designed to capture physical, psychological, and social problems in accordance with the bio-psycho-social model to more completely describe patients’ problems. Again, it is the structure of ICPC – encompassing the full process of care over time – that enhances its value.
Finally, we strongly disagree with the authors’ assertion that revisions underway in ICD, ICPC and SNOMED-CT “indicate either problems of structure or problems of content”. The primary challenge we face in improving healthcare information technology is finding the proper balance between usability and comprehensiveness. Each of these classifications/terminologies is evolving, but we can now see how they can be integrated: ICPC to provide structure and facilitate data aggregation, ICD and SNOMED-CT to provide fine grained classification and terminology. But in practical terms, there will be a continuing need for good clinical interface terminologies at the user interface. GPs tend to spend as little time as possible on diagnostic coding, and interface terminologies are at this point a necessary compromise. The actual search methodology that is used also has a strong effect on the usability and accuracy of the coding. However, the interface terminology has to be large enough to provide a high specificity. The authors imply that the interface terminology used in Norway is a large terminology but at 6390 terms it is relatively limited compared to the interface terminologies used in e.g. the Netherlands (94,630 search texts). It is not clear whether the problems seen in this study were the result of the relatively small size of the interface terminology or the tendency of GPs to choose their own language for diagnosis, but they were clearly not caused by the structure of ICPC.
Marianne Rosendal, Laurent Letrilliart, Kees van Boven, Anders Grimsmo, Helena Britt, Jean Karl Soler, Marc Verbeke, Marc Jamoulle, Ray Simkus, and Mike Klinkman (WICC chair)
Reference List
(1) Botsis T, Bassoe CF, Hartvigsen G. Sixteen years of ICPC use in Norwegian primary care: looking through the facts. BMC Med Inform Decis Mak 2010; 10:11.
(2) Soler JK, Okkes I, Wood M, Lamberts H. The coming of age of ICPC: celebrating the 21st birthday of the International Classification of Primary Care. Fam Pract 2008; 25(4):312-317.
(3) Letrilliart L., Bacis A.K., Mennerat F., et al. Interface Terminologies: A Case Study on the International Classification of Primary Care. World Academy of Science, Engineering and Technology 2009; 54:614-617.
(4) Loh A., Zelmer J. Wonca and IHTSDO Seek to Strengthen Informatics in Primary Care. Launch of Cooperative Effort to Better Serve Family Doctors and their Patients. www ihtsdo org/fileadmin/user_upload/Docs_01/Press_Releases/Wonca_IHTSDO_agreement_announcement pdf [ 2010
(5) Hsia DC, Ahern CA, Ritchie BP, Moscoe LM, Krushat WM. Medicare reimbursement accuracy under the prospective payment system, 1985 to 1988. JAMA 1992; 268(7):896-899.
(6) Nickelsen TN. [Data validity and coverage in the Danish National Health Registry. A literature review]. Ugeskr Laeger 2001; 164(1):33-37.
Competing interests
The authors are members of the Wonca International Classification Committee, which develops the ICPC classification.