The Visual Localization Benchmark

Visual localization is the problem of estimating the 6 Degree-of-Freedom (DoF) camera pose from which a given image was taken relative to a reference scene representation. Visual localization is a key technology for applications such as Augmented, Mixed, and Virtual Reality, as well as for robotics, e.g., for self-driving cars. In order to evaluate visual localization over longer periods of time, we provide benchmark datasets aimed at evaluating 6 DoF pose estimation accuracy over large appearance variations caused by changes in seasonal (summer, winter, spring, etc.) and illumination (dawn, day, sunset, night) conditions. Each dataset consists of a set of reference images, together with their corresponding ground truth poses, and a set of query images. A triangulated 3D model is provided for each dataset and can be used by structure-based localization approaches. To ensure fairness and comparability of results, the reference poses for the query images is withheld and we provide an evaluation service to measure pose accuracy.

Benchmark highlights:

New challengesQuery images taken under different seasonal and illumination conditions
Varied datasets Datasets taken with hand-held cameras and from vehicle-mounted cameras, covering various geographical locations
Unified evaluation protocol Both the position and orientation accuracy are taken into account.

Updates

2019-04-15: Submissions for the InLoc dataset and the SILDa Weather and Time of Day datasets are now available.