In this study, we extracted information from approximately 400,000 OCT report files. The overall accuracy of Value Reader was 99.67%. The results generated by Value Reader and Finding Classifier were stored in CSV files, which were subsequently imported into the OMOP CDM easily. We also plan to load cropped tomographic images into the PACS as soon as possible.
Several studies that used open-source or commercially available OCR software to extract text from optical coherence tomography have been described. [15,16,17] Aptel et al. used a commercial software, ABBYY FineReader v 9.0 (Avanquest Software, LaGarenne Colombes, France), and Demea et al. used NI Vision Assistant and NI Vision Development Module (National Instrument, Austin, Texas, USA) [15, 16]. Sood et al. [17] used Tesseract, an open-source OCR engine. The researchers used their OCR modules to extract only one or two types of research-specific data. They could not structure the data for general purposes. The development of customized and flexible software providing standardized, structured, and interoperable information from various ophthalmic examination devices has not been previously reported.
Report image files are simply an array of pixels unless a clinician manually reviews them. It is essential for clinicians to base decisions on comparisons of test results, however current report image files make this process difficult. The new software described in this paper enables the extraction of information from image files written in a non-standardized fashion and subsequently standardizes it in a structured CSV file. It can accommodate various types of image data and makes it easier for clinicians to interpret by organizing data. Structured information may be used in various ways; for example, an ophthalmologist may use it to interpret disease activity or progression by comparing measurement values or by referring to an automatically generated result. These features aid clinicians in decision-making without delays or errors. Processed data such as serial changes in macular thickness, macular volume, and disease activity of neovascular age-related macular degeneration in Fig. 2 can help ophthalmologists reduce misdiagnosis and determine appropriate timing of treatment. However, calibration among values may be considered in cases where the measurement values are different despite examining the same target. (e.g., devices from multiple vendors or upgraded measurement technique).
Deep learning has the potential to revolutionize medicine as demonstrated by a recently developed model with good performance for classification of pathologic findings [2]. In terms of accuracy, previous studies did not address the accuracy of their OCR modules [15,16,17]. However, there is a limit to improving accuracy by using commercial software or open-source library in our experience. For this reason, we developed an OCR engine using deep learning. It is possible to achieve the high accuracy required in the medical field by using deep learning for optical character recognition, since new data can raise its accuracy continuously. In addition, it is vital that these types of models are applicable to real-world practice and that they address unmet clinical needs such as trend analysis or prediction of future disease activity. In this report, we showed how several algorithms implemented in Value Reader and Finding Classifier modules with deep learning enabled a real-world clinician to make practical decisions by providing pathologic findings or evidence of disease activity.
Numerous clinical data registries containing sizeable real-world data sets have recently been established [18]. In addition, OMOP CDM that allows systematic analysis by transforming and integrating clinical data into standardized formats are emerging as one of the most important approaches in medical research [19, 20]. If the large amount of data held in millions of report image files could be extracted and entered into the databases mentioned above using our newly developed software, the resulting datasets might become some of the most powerful clinical tools available worldwide. The reliability and reproducibility of studies can also be improved tremendously because researchers can easily build databases containing millions of data in standardized form with this software. We completed the data extraction from OCT reports and loaded them into the OMOP CDM at our hospital. However, cropped tomographic images have not yet been incorporated into the PACS due to hospital policy. As with other medical images, encapsulated tomographic images in the DICOM format will eventually be uploaded into the PACS. This process will provide a structure within the PACS enabling analysis of multi-modal images, and will expand the base for imaging studies.
Full-scale validation of the Finding Classifier was not performed since it was beyond the scope of our study. In addition, since permission by government authorities is required to employ the Finding Classifier in clinical practice, we only investigated its potential application. We are currently in the process of building a ground-truth dataset utilizing an in-house database and an increasing number of open databases, and therefore we expect that Finding Classifier will soon be made more reliable.
Our software has several limitations. First, it is difficult to convert characters with low resolution, although humans can read them relatively easily. Second, our software can organize data for large-scale retrospective research, but it lacks real-time function for practice. It is necessary to build a real-time extraction, transformation, and loading pipeline and to integrate it with an EHR system for clinical implementation. Third, we need to validate and further develop the Finding Classifier. Finally, other considerations such as calibration among the same type of measurements derived from different vendors are required.