Standardizing Heterogeneous Annotation Corpora Using HL7 FHIR for Facilitating their Reuse and Integration in Clinical NLP

Manually annotated clinical corpora are commonly used as the gold standards for the training and evaluation of clinical natural language processing (NLP) tools. The creation of these manual annotation corpora, however, is both costly and time-consuming. There is an emerging need in the clinical NLP community for reusing existing annotation corpora across different clinical NLP tasks. The objective of this study is to design, develop and evaluate a framework and accompanying tools to support the standardization and integration of annotation corpora using the HL7 Fast Healthcare Interoperability Resources (FHIR) specification. The framework contains two main modules: 1) an automatic schema transformation module, in which the annotation schema in each corpus is automatically transformed into the FHIR-based schema; 2) an expert-based verification and annotation module, in which existing annotations can be verified and new annotations can be added for new elements defined in FHIR. We evaluated the framework using various annotation corpora created as part of different clinical NLP projects at the Mayo Clinic. We demonstrated that it is feasible to leverage FHIR as a standard data model for standardizing heterogeneous annotation corpora for their reuse and integration in advanced clinical NLP research and practices