Combining Deep Learning and Knowledge-driven Reasoning for Chest X-Ray Findings Detection

Abstract

Application of deep learning algorithms in medical imaging analysis is a fascinating and growing research area. While deep learning methods are thriving in the medical domain, they seldom utilize the rich knowledge associated with connected radiology reports. The radiology reports are a great source of knowledge, and the knowledge derived from these reports can enhance deep learning models' performance. In this work, we developed a comprehensive chest X-ray findings' vocabulary that is used to automatically annotate an extensive collection of chest X-rays using associated radiology reports and a vocabulary-driven concept annotation algorithm . The annotated X-rays are used to train the deep learning module's neural network architecture for finding detection. Finally, we developed a knowledge-driven reasoning algorithm that leverages knowledge learnt from X-ray reports to improve the deep learning module's performance on the finding detection. The reasoning algorithm significantly improves upon the deep learning module performance with 9.09 % improvement in F1-score.