The Québec BCG Vaccination Registry (1956–1992): assessing data quality and linkage with administrative health databases

Organization of the Québec BCG Vaccination Registry

Vaccination certificates covering the years between 1926 and 1955 inclusively were only available on microfilms. Beginning in the 1960s, clerical staff manually entered the information from vaccination certificates dated 1956 onwards to create listings for consultation purposes. Eight series of alphabetically sorted paper listings were produced, each covering a few calendar years from 1956 to 1992. This represented 140 volumes totalling over 60,000 pages (~4.2 million lines of data), which, along with 123 microfilm rolls, constituted the registry’s searchable hard-copy. In the paper listings, information corresponding to an individual event was summarized on one line that specifically contained information on surname and given name, father’s given name, birth date, sex, tuberculin reactivity test (type, date, and reaction in millimetres), current vaccination (date and administration mode), date of prior vaccinations, and medical institution code.

Vaccination certificates (microfilms and paper) and paper-based listings of the BCG Vaccination Registry have been converted to an electronic format in 2010. Computerization, performed by Trigonix Inc. (Montreal, Canada), comprised graphical imaging of vaccination certificates (microfilms and paper) and transfer of paper listings into a searchable electronic database using optical character recognition. In a first step, this involved scanning each page within the registry’s paper listings to create PDF (Portable Document Format) files. In a second step, the optical character recognition parameters were optimized for each of the eight series of paper listings due to their differing formats, thereby allowing their conversions into an electronic database.

Data quality

The quality and accuracy of the electronic version of the Québec BCG Vaccination Registry, hereafter referred to as the electronic BCG Registry, was ascertained using two distinct verification samples. Sample size was determined based on having a feasible number of records to retrieve and review, and achieving a reasonable statistical precision. Firstly, we determined agreement between paper listings and the electronic BCG Registry to assess accuracy of the computerization process. Individual records (n = 5,268) from paper listings (1956–1992) were sampled by systematically selecting pages within volumes, and lines within pages to roughly represent 0.1% from each of the eight series. Secondly, we determined agreement between vaccination certificates and the electronic BCG Registry to document the accuracy and completeness of the electronic database compared with its archived raw data. Vaccination certificates, stored in archive filing cabinets, are organized by year and geographical region where vaccination took place. We systematically selected 4,972 vaccination certificates (~0.1% of the registry), approximately 250 certificates for each of the 20 years of interest (1956–1975) sampled from all available geographical regions.

For the two samples, the quality control and verification process was documented in terms of the completeness of information and presence of discrepancies. Information was compared along several personal nominal identifiers and BCG-related variables. We computed, for each variable, percent agreement and 95% confidence intervals (CIs) among records containing valid information (non-empty fields). The proportion of records with complete agreement on all variables was also calculated. Verification and analyses were respectively carried out with Filemaker Pro, version 11.0 (FileMaker Inc., Santa Clara, California) and SPSS, version 17.0 (SPSS Inc., Chicago, Illinois).

Linkage with administrative databases

To assess linkage feasibility, we determined the proportion of successful record linkages with provincial demographic and administrative medical databases, overall and per birth year. From the electronic BCG Registry, 3,500 subjects were randomly sampled among 491,861 individuals born from 1961 to 1974 (~250 per birth year) and vaccinated between 1970 and 1974. The nominal information extracted included child’s surname, given names of the child and father, sex, and date of birth. Data linkage was independently performed between the electronic BCG Registry and both the Birth Registry, administered by the Institut de la statistique du Québec (ISQ), and the Healthcare Registration File (Régie de l’assurance maladie du Québec, RAMQ). The Birth Registry provides information on all births and stillbirths occurring in the province as well as perinatal and parental sociodemographic data, retrieved from birth certificates. The RAMQ is the government body responsible for administration of health care services in Québec. Our linkage was conducted with the Healthcare Registration File which includes, for all beneficiaries of the universal public health system, information such as birth date, sex, postal code and year of death (if applicable).

Probabilistic record linkages between the electronic BCG Registry and the Birth Registry were done by the Environnement pour la promotion de la santé et du bien-être (EPSEBE) team at ISQ [12] using five basic identifiers (child’s surname and given name, date of birth, sex, initial of father’s given name). Deterministic linkages between the electronic BCG Registry and the Healthcare Registration File were carried out by RAMQ using the same identifiers, except for using the full father’s given name instead of his initial. For matching purposes, nominal information was independently standardized by ISQ and RAMQ in several ways (e.g., capitalizing all letters, eliminating general identifiers such as “Ms.” or “unknown”, standardizing date formats, removing blank spaces, splitting hyphenated names into several data fields). This entire project, including the transfer of confidential data, was done in accordance with the province of Québec’s legal and ethical requirements. Procedures, data access, and ethical issues were assessed and approved by the Commission d’accès à l’information (reference number: 11 02 67 (10 08 48, 09 08 39)), INRS Research Ethics Committee (reference number: CER-09-203), and ISQ (reference number: KB2-Rousseau-pilote, 09–08).

The presence of certain potential confounders pertinent to investigating hypotheses on a link between BCG vaccination and chronic diseases was verified, given that a typical limitation of using administrative databases for research purposes is the lack of information on such variables. Some useful variables documented in the Birth Registry over the years covered include gestational age, birth weight, number of older siblings, mother’s municipality of residence, as well as parents’ age and birthplace.

Data quality

In the present study, we have assessed and documented data accuracy and completeness of the electronic Québec BCG Registry. This provided us with an opportunity to correct some systematic errors that have occurred during the computerization process, although very high agreement was observed between paper listings and the final electronic BCG Registry. Very few records (0.8%) had discrepancies in more than 1 variable upon comparing computerized records to their corresponding vaccination certificates. For discrepancies observed in given name and sex, general algorithms could then be applied to remove assigned generic names and verify the sex variable in the electronic BCG Registry before conducting further linkages with administrative databases.

Only one study was previously carried out to validate the Québec BCG Registry, comparing paper listings to childhood vaccination booklets [13]. Based on 369 subjects with both sources of information, the overall agreement was 78% (kappa = 0.56). A noteworthy limitation is that the vaccination booklet cannot be considered as a gold standard, since it might not have been provided by parents every time a vaccination took place.

Linkage with administrative databases

Our results demonstrate that linkage of the electronic BCG Registry to administrative databases towards establishing a retrospective cohort study is highly feasible. However, in the absence of a unique identifier, the linkage method and missing values in personal identifiers had an impact on linkage success across birth years.

Approximately 60% of the electronic BCG Registry sample was linked to both the Birth Registry and the Healthcare Registration File. Successful overall linkage rates were much lower with the latter due to the use of a deterministic method, which did not allow for flexibility with regards to the initial quality of data or presence of missing values. Thus, subjects with missing information on any of the following four variables - child’s surname, child’s given name, sex, and date of birth - could not be matched. These variables were considered as absolutely necessary for linkage, while father’s name was optional. Information on given name was missing from the BCG registry when the vaccine was received very shortly after birth, presumably before the child’s name was chosen by parents. For linkage purposes, subjects were selected if they had received the vaccine between 1970 and 1974. Hence, there were no missing given names among those born from 1961 to 1968, as these children were 2 to 14 years old at the time of vaccination. Given name was missing from the BCG Registry for few subjects (4%) born in 1969, for 19.6% of those born in 1970, and up to 29.2% among subjects born in 1974. This artefact, related to our selection criteria and the use of a deterministic method, could explain the lower linkage rates (ranging from 54% to 59%) with the Healthcare Registration File among those born after 1969. As expected, the probabilistic linkage method performed better, except for those born in the early 1960s, likely due to the lower quality of information from the Birth Registry at that time. The quality of information compiled in the Birth Registry is generally considered to be best from 1975 onwards, but had likely started to improve in prior years. This was evidenced by the highest linkage rate (87.2%) achieved with the Birth Registry for the 1974 birth year.

The non-negligible proportion (12.5%) of subjects linked solely with the Healthcare Registration File could be due to the fact that the Birth Registry contains information only on those born in Québec. One foreseeable situation is that a child born outside of Québec who subsequently immigrated to Québec would be present in the Healthcare Registration File and the BCG Registry, but definitely not in the Birth Registry.

Use of the electronic BCG Registry for research purposes

Although BCG vaccination in Québec was discontinued in the 1970s except for selected high risk groups (Inuit and many on-reserve First Nations infants in higher tuberculosis incidence communities as well as health workers) [14, 15], scientifically pertinent research hypotheses could be addressed using the information available in the BCG Vaccination Registry. Before its computerization in 2010, the Québec BCG Registry had been used in research on tuberculosis [13, 16, 17], leukaemia [18, 19], type 1 diabetes [20], and asthma [21]. Although data from this registry can be useful in various epidemiological study designs, their most efficient utilization will be through linkage with existing provincial medical administrative health databases. Such methodology will enable studying intended and unintended effects of BCG vaccination in a population-based setting. Our research team is particularly interested in investigating the impact of early life non-specific stimulation of the immune function, as resulting from BCG vaccination, on the prevention or development of inflammatory and auto-immune diseases in childhood [22, 23].

Depending on the specific research questions investigated in future studies, variables documented in the Birth Registry may be used to control for confounding. In this sense, our finding that this information is present from birth year 1965 onwards will be extremely useful when planning future studies.

A limitation worth mentioning is that adverse events following immunization were not routinely compiled in the BCG Vaccination Registry, unfortunately impeding population-based research on this pertinent issue.

In Québec, there are no unique personal identification numbers (PINs) for citizens to allow unambiguous linkage between public registries. A particular strength of our study was thus the ability to carry out linkage despite the absence of a unique common identifier. Instead, we relied on identification variables available in the BCG and Birth Registries, as well as in the Healthcare Registration File. We demonstrated the feasibility of this approach and suggested avenues to pursue, such as probabilistic as opposed to deterministic linkage, when using historical data coupled with administrative databases. For future epidemiological studies, linkage with provincial medical databases will allow obtaining data from physician billing claims, prescription drugs claims, and hospitalizations.

The present study was crucial to the future use of a largely unexploited and comprehensive vaccination dataset for research purposes. Moreover, documentation of the various steps undertaken towards validation and linkage of several databases will provide important methodological clues to researchers aiming at assembling similar research infrastructures in other settings.