The Visual Localization Benchmark
Visual localization is the problem of estimating the 6 Degree-of-Freedom (DoF) camera pose from which a given image was taken relative to a reference scene representation. Visual localization is a key technology for applications such as Augmented, Mixed, and Virtual Reality, as well as for robotics, e.g., for self-driving cars. In order to evaluate visual localization over longer periods of time, we provide benchmark datasets aimed at evaluating 6 DoF pose estimation accuracy over large appearance variations caused by changes in seasonal (summer, winter, spring, etc.) and illumination (dawn, day, sunset, night) conditions. Each dataset consists of a set of reference images, together with their corresponding ground truth poses, and a set of query images. A triangulated 3D model is provided for each dataset and can be used by structure-based localization approaches. To ensure fairness and comparability of results, the reference poses for the query images is withheld and we provide an evaluation service to measure pose accuracy.
|New challenges||Query images taken under different seasonal and illumination conditions|
|Varied datasets||Datasets taken with hand-held cameras and from vehicle-mounted cameras, covering various geographical locations|
|Unified evaluation protocol||Both the position and orientation accuracy are taken into account.|
2020-06-18: Enabled submission to the RobotCar-Seasons v2 dataset. This consists of the same database and query images as the original RobotCar-Seasons dataset, but uses a different test-train split.
2020-06-18: The Aachen Day-Night v 1.1 dataset has been added. This is an extension of the Aachen Day-Night dataset that contains additional night-time queries. Please refer to the corresponding arXiv paper for more information. Additionally, the reference poses for the night-time query images in the original Aachen Day-Night dataset have been updated to more accurate poses. The results in the Aachen Day-Night table may therefore have changed. Since we consider the new night-time poses to be more accurate, we have also changed the error thresholds to be the same as the ones for daytime.
2020-01-21: Added the Symphony Seasons dataset
2019-04-15: Submissions for the InLoc dataset and the SILDa Weather and Time of Day datasets are now available.