Comparing the effects of two semantic terminology models on classification of clinical notes: A study of heart murmur findings

Objectives: We compared the effects of two semantic terminology models on classification of clinical notes through a study in the domain of heart murmur findings. Methods: One schema was established from the existing SNOMED CT model (S-Model) and the other was from a template model (T-Model) which uses base concepts and non-hierarchical relationships to characterize the murmurs. A corpus of clinical notes (n=309) was collected and annotated using the two schemas. The annotations were coded for a decision tree classifier for text classification task. The standard information retrieval measures of precision, recall, f-score and accuracy and the paired t-test were used for evaluation. Results: The performance of S-Model was better than the original T-Model (p<0.05 for recall and f-score). A revised T-Model by extending its structure and corresponding values performed better than S-Model (p<0.05 for recall and accuracy). Conclusion: We discovered that content coverage is a more important factor than terminology model for classification; however a templatestyle facilitates content gap discovery and completion.