Concept-oriented access to longitudinal multimedia medical records: A case study in brain tumor patient management

Abstract

The current clinical practice requires physicians to gather, interpret and correlate information from multiple independent multimedia data sources to manage patients. Due to poor structuring and organization, it is too timeconsuming to access the information snippets embedded in the various pieces of data in the longitudinal patient records. This becomes more of a problem when correlating the temporal progression of various factors obtained from patients clinical, laboratory, imaging and genomics studies. Making such correlations is an essential component of the prognosis and treatment planning tasks in patient care. In addition, the similarities in the disease progression pattern among different patients and their relationships to outcomes remain hidden from the clinicians in the piecemeal use of the data. We believe that there is a gap between the decision-enabling information and insight required for efficient patient management and the heterogeneous data comprising the patient records that can be bridged with advanced multimodal content analytics, semantic information organization, summarization, and visualization tools. In this paper we present a case study in organizing, accessing, and visualizing information obtained through structuring the multimedia and multimodal data for brain tumor patient management and how such information map to the needs of the clinicians. We report our early work on the analytics, user interface and the preliminary evaluation results which indicate that the presented approach caters well to the clinician needs for the task of brain tumor patient management.