Multi Cost Function Fuzzy Stereo Matching Algorithm for Object Detection and Robot Motion Control

Akhil Appu Shetty, Navya Thirumaleshwar Hegde, Aldrin Claytus Vaz, C. R. Srinivasan

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)


Stereo matching algorithms work with multiple images of a scene, taken from two viewpoints, to generate depth information. Authors usually use a single matching function to generate similarity between corresponding regions in the images. In the present research, the authors have considered a combination of multiple data costs for disparity generation. Disparity maps generated from stereo images tend to have noisy sections. The presented research work is related to a methodology to refine such disparity maps such that they can be further processed to detect obstacle regions. A novel entropy based selective refinement (ESR) technique is proposed to refine the initial disparity map. The information from both the left disparity and right disparity maps are used for this refinement technique. For every disparity map, block wise entropy is calculated. The average entropy values of the corresponding positions in the disparity maps are compared. If the variation between these entropy values exceeds a threshold, then the corresponding disparity value is replaced with the mean disparity of the block with lower entropy. The results of this refinement are compared with similar methods and was observed to be better. Furthermore, in this research work, the v-disparity values are used to highlight the road surface in the disparity map. The regions belonging to the sky are removed through HSV based segmentation. The remaining regions which are our ROIs, are refined through a u-disparity area-based technique. Based on this, the closest obstacles are detected through the use of k-means segmentation. The segmented regions are further refined through a u-disparity image information-based technique and used as masks to highlight obstacle regions in the disparity maps. This information is used in conjunction with a kalman filter based path planning algorithm to guide a mobile robot from a source location to a destination location while also avoiding any obstacle detected in its path. A stereo camera setup was built and the performance of the algorithm on local real-life images, captured through the cameras, was observed. The evaluation of the proposed methodologies was carried out using real life out door images obtained from KITTI dataset and images with radiometric variations from Middlebury stereo dataset.

Original languageEnglish
Pages (from-to)356-370
Number of pages15
JournalJournal of Robotics and Control (JRC)
Issue number3
Publication statusPublished - 05-2023

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Artificial Intelligence


Dive into the research topics of 'Multi Cost Function Fuzzy Stereo Matching Algorithm for Object Detection and Robot Motion Control'. Together they form a unique fingerprint.

Cite this