Visual Concept Naming: Discovering Well-Recognized Textual Expressions of Visual Concepts

Abstract

We propose a task called Visual Concept Naming to associate visual concepts with the corresponding textual expressions, i.e., names of visual concepts found in real-world multimodal data. To tackle the task, we create a dataset consisting of 3.4 million tweets in total in three languages. We also propose a method for extracting candidate names of visual concepts and validating them by exploiting Web-based knowledge obtained through image search. To demonstrate the capability of our method, we conduct an experiment with the dataset we create and evaluate names obtained by our method through crowdsourcing, where we establish an evaluation method to verify the names. The experimental results indicate that the proposed method can identify a wide variety of names of visual concepts. The names we obtained also show interesting insights regarding languages and countries where the languages are used.1