Sketch-Based Image Retrieval Using Convolutional Neural Networks Based on Feature Adaptation and Relevance Feedback

Sketch-based Image Retrieval (SBIR) is an approach where natural images are retrieved according to the given input sketch query. SBIR has many applications, for example, searching for a product given the sketch pattern in digital catalogs, searching for missing people given their prominent features from a digital people photo repository etc. The main challenge involved in implementing such a system is the absence of semantic information in the sketch query. In this work, we propose a combination of image prepossessing and deep learning-based methods to tackle this issue. A binary image highlighting the edges in the natural image is obtained using Canny-Edge detection algorithm. The deep features were extracted by an ImageNet based CNN model. Cosine similarity and Euclidean distance measures are adopted to generate the rank list of candidate natural images. Relevance feedback using Rocchio’s method is used to adapt the query of sketch images and feature weights according to relevant images and non-relevant images. During the experimental evaluation, the proposed approach achieved a Mean average precision (MAP) of 71.84%, promising performance in retrieving relevant images for the input query sketch images.