Possibility to use product image and review text based on the association between onomatopoeia and texture

With the widespread use of online shopping in recent years, consumer search requests for products have become more diverse. Previous web search methods have used adjectives as input by consumers. However, given that the number of adjectives that can be used to express textures is limited, it is debatable whether adjectives are capable of richly expressing variations of product textures. In Japanese, tactile and visual textures are easily and frequently expressed by onomatopoeia, such as“ fuwa-fuwa” for a soft and light sensation and“ kira-kira” for a glossy texture. Onomatopoeia are useful for understanding not only material textures but also a user’s intuitive, sensitive, and even ambiguous feelings evoked by materials. In this study, we propose a system to rank FMD images corresponding to texture associated with Japanese onomatopoeia based on their symbolic sound associations between the onomatopoeia phonemes and the texture sensations. Our system quantitatively estimates the texture sensations of input onomatopoeia, and calculates the similarities between the users’ impressions of the onomatopoeia and those of the images. Our system also suggests the images which best match the input onomatopoeia. An evaluation of our method revealed that the best performance was achieved when the SIFT features, the colors of the images, and text describing impressions of the images were used.