GIS-Assisted Object Detection and Geospatial Localization (ECCV 2014 - Download PDF,)
- One Minute Summary Video
- Introduction
- Our Approach
- Results
- Dataset and DPM Models
- Related Publications
One Minute Summary Video
Introduction
Geographical Information System (GIS) databases contain information about several objects, such as traffic signals, road signs, fire hydrants, in many urban areas. We propose a method for improving object detection using a set of priors acquired from GIS databases. Furthermore, we propose that the GIS objects can be used as cues for discovering the location where an image was taken.
Our Approach
Given a database of object locations from GIS and a query image with metadata, we project the objects onto the image plane.
We also apply object detection on the query image using DPM and obtain a set of candidate bounding boxes for the objects. Then, we fuse the GIS priors with the potential detections to find the final object bounding boxes. In order to cope with various inaccuracies and practical complications, such as noisy metadata, occlusion, inaccuracies in GIS, and poor candidate bounding boxes, we formulate our fusion as a higher order graph matching problem which we robustly solve using RANSAC. We demonstrate that this approach outperforms well established object detectors, such as DPM, with a large margin.
For geospatial localization, our hypothesis is based on the idea that the objects visible in one image, along with their relative spatial location, provide distinctive cues for the geo-location. In order to estimate the geo-location based on the generic objects, we perform a search on a dense grid of locations over the covered area. We assign a score to each location based on the similarity of its GIS objects and the imperfect object bounding boxes in the image.
We demonstrate that over a broad urban area of over 10 square kilometers, this semantic approach can significantly narrow down the localization search space, and occasionally, even find the correct location.
Results
The following figure illustrates the improvement in the precision recal curves of five different object detection classes:
The following shows different steps of our method for three different examples:
The following figure shows our geo-localization results, (a) shows the effect of leveraging the geometric structure of the objects, (b) shows the effect of the number of objects being used and (c) shows the effect of each object class in the geo-localization task :
The figure below illustrates four different examples of geo-localization using our method.
Dataset and DPM Models
Download GIS Dataset for Objects and Building Outlines,
Download DPM Models (for Traffic Sign, Traffic Signal, and Street Lights) ,
Related Publications
Shervin Ardeshir, Amir Roshan Zamir, Alejandro Torroella, and Mubarak Shah, "GIS-Assisted Object Detection and Geospatial Localization", in Proceedings of IEEE European Conference on Computer Vision (ECCV), September 2014 [PDF | Project Page]
Take a look at a few of our other papers and projects in the area of geo-spatial analysis of images and videos:
Amir Roshan Zamir, Shervin Ardeshir and Mubarak Shah, "Robust Refinement of GPS-Tags using Random Walks with an Adaptive Damping Factor", in Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), June 2014 [PDF | Project Page]
Amir Roshan Zamir, Afshin Dehghan and Mubarak Shah, "Visual Business Recognition - A Multimodal Approach", In Proceeding of ACM International Conf. on Multimedia (ACM MM), 2013 [PDF | BibTeX | Supplemental Mat.]
Take a look at a few of our other papers and projects in the area of geo-spatial analysis of images and videos:
Amir Roshan Zamir, Shervin Ardeshir and Mubarak Shah, "Robust Refinement of GPS-Tags using Random Walks with an Adaptive Damping Factor", in Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), June 2014 [PDF | Project Page]
Amir Roshan Zamir and Mubarak Shah, "Image Geo-localization Based on Multiple Nearest Neighbor Feature Matching using Generalized Graphs", In IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2014 [PDF | Project Page]
Gonzalo Vaca, Amir Roshan Zamir and Mubarak Shah, "City Scale Geo-spatial Trajectory Estimation of a Moving Camera", in Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), June 2012 [PDF | BibTeX| Project Page]
Amir Roshan Zamir and Mubarak Shah, "Accurate Image Localization Based on Google Maps Street View", In Proceedings of European Conference on Computer Vision (ECCV), 2010 [PDF | BibTeX | Project Page]
