Workshop on Long-Term Visual Localization under Changing Conditions
We will be hosting a long-term visual localization workshop at CVPR 2019, which will contain competitions for three different visual localization scenarios. Please see the main workshop page for details.
Results can be submitted to the challenge under the Submission tab above. Select the relevant challenge from the dropdown menu. For the visual localization challenge, and the end-to-end localization challenge, please submit the method once per dataset, using the same method name for all submissions. Datasets for which no results are available for a method will be displayed with zeroes.
The ranking is performed using the Shultze method.
The pose error thresholds which have been used for each dataset to calculate the fraction of correctly localized images are listed for each dataset on the Benchmark page.
Listed below are the public results on the three benchmark datasets.
Visual localization challenge
Method | Aachen | Extended CMU Seasons | RobotCar Seasons | InLoc | SILDa | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
day | night | urban | suburban | park | day all | night all | duc1 | duc2 | evening | snow | night | |
Hierarchical-Localization (multi-camera when available) | 80.5 / 87.4 / 94.2 | 42.9 / 62.2 / 76.5 | 91.6 / 96.4 / 99.1 | 84.7 / 91.5 / 98.6 | 69.3 / 77.8 / 90.5 | 53.8 / 80.4 / 96.0 | 11.2 / 27.7 / 49.1 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
Visual Localization Using Sparse Semantic 3D Map | 71.8 / 91.5 / 96.8 | 40.8 / 63.3 / 80.6 | 88.8 / 93.6 / 96.3 | 78.0 / 83.8 / 89.2 | 63.6 / 70.3 / 77.3 | 54.5 / 81.6 / 96.7 | 12.3 / 28.5 / 46.5 | 41.4 / 59.1 / 71.2 | 38.2 / 49.6 / 58.0 | 29.4 / 66.5 / 93.4 | 2.2 / 11.1 / 57.4 | 20.4 / 41.8 / 69.0 |
Hierarchical-Localization NetVLAD+SuperPoint | 80.5 / 87.4 / 94.2 | 42.9 / 62.2 / 76.5 | 89.5 / 94.2 / 97.9 | 76.5 / 82.7 / 92.7 | 57.4 / 64.4 / 80.4 | 53.1 / 79.1 / 95.5 | 7.2 / 17.4 / 34.4 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
Asymmetric Hypercolumn Matching | 47.8 / 72.2 / 91.3 | 30.6 / 53.1 / 78.6 | 65.7 / 82.7 / 91.0 | 66.5 / 82.6 / 92.9 | 54.3 / 71.6 / 84.1 | 45.7 / 78.0 / 95.1 | 22.3 / 61.8 / 94.5 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
Localizing Visual Landmarks for 2D matching | 62.4 / 71.8 / 79.9 | 24.5 / 35.7 / 44.9 | 84.3 / 89.3 / 93.0 | 68.0 / 75.1 / 84.4 | 42.4 / 51.4 / 69.7 | 51.1 / 77.7 / 92.3 | 13.8 / 30.3 / 57.7 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
DGCNCCC | 22.9 / 49.8 / 84.7 | 19.4 / 37.8 / 68.4 | 17.1 / 41.5 / 89.1 | 8.9 / 26.8 / 77.1 | 4.8 / 16.2 / 63.3 | 7.2 / 28.3 / 87.6 | 0.0 / 1.1 / 13.4 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
DenseVLAD | 0.0 / 0.1 / 22.8 | 0.0 / 2.0 / 14.3 | 14.7 / 36.3 / 83.9 | 5.3 / 18.7 / 73.9 | 5.2 / 19.1 / 62.0 | 7.6 / 31.2 / 91.2 | 1.0 / 4.4 / 22.7 | 0.0 / 1.5 / 5.1 | 0.0 / 0.8 / 2.3 | 0.2 / 6.2 / 42.1 | 0.0 / 0.0 / 0.7 | 0.0 / 2.6 / 53.9 |
COCL_HR | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 12.7 / 60.6 / 75.4 | 0.5 / 1.6 / 2.2 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
ONavi-H | 77.7 / 89.0 / 95.8 | 49.0 / 65.3 / 85.7 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
SemLocMet | 76.6 / 88.6 / 95.1 | 40.8 / 63.3 / 78.6 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
Cascaded Parallel Filtering for Memory-Efficient Image-based Localization | 76.7 / 88.6 / 95.8 | 25.5 / 38.8 / 54.1 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
Local feature challenge
Method | Aachen |
---|---|
night | |
D2-Net - single-scale | 45.9 / 68.4 / 88.8 |
R2D2 V2 20K | 46.9 / 66.3 / 88.8 |
R2D2 10k keypoints | 45.9 / 66.3 / 88.8 |
densecontextdesc10k_upright_mixedmatcher_v2 | 48.0 / 63.3 / 88.8 |
densecontextdesc10k_upright_mixedmatcher | 46.9 / 65.3 / 87.8 |
D2-Net - single-scale off-the-shelf | 41.8 / 69.4 / 86.7 |
DELF - new model | 39.8 / 61.2 / 85.7 |
DELF - old model | 39.8 / 60.2 / 84.7 |
saliency_ranking_net (multi-scale) | 44.9 / 59.2 / 77.6 |
SuperPoint (baseline) | 42.9 / 57.1 / 77.6 |
densecontextdesc10k_upright | 41.8 / 57.1 / 79.6 |
contextdesc10_upright | 40.8 / 55.1 / 80.6 |
HesAffNet-HardNet2 | 37.8 / 54.1 / 75.5 |
m2_real | 34.7 / 50.0 / 66.3 |
Upright RootSIFT (Feature Challenge Baseline) | 33.7 / 52.0 / 65.3 |
m2 | 27.6 / 42.9 / 60.2 |
ELF | 13.3 / 21.4 / 30.6 |
End-to-end localization challenge
Method | Aachen | Extended CMU Seasons | RobotCar Seasons | InLoc | SILDa | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
day | night | urban | suburban | park | day all | night all | duc1 | duc2 | evening | snow | night | |
Localizing Visual Landmarks for Place Recognition | 0.0 / 0.2 / 20.8 | 0.0 / 1.0 / 10.2 | 17.3 / 42.5 / 89.0 | 5.8 / 19.4 / 76.1 | 6.6 / 23.1 / 73.0 | 7.9 / 30.0 / 85.9 | 4.1 / 15.7 / 59.1 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.3 / 7.1 / 41.0 | 0.0 / 0.0 / 0.9 | 0.0 / 2.7 / 56.9 |
NetVLAD | 0.0 / 0.2 / 18.9 | 0.0 / 2.0 / 12.2 | 12.2 / 31.5 / 89.8 | 3.7 / 13.9 / 74.7 | 2.6 / 10.4 / 55.9 | 6.4 / 26.3 / 90.9 | 0.3 / 2.3 / 15.9 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
joint image segemation and depth to posenet | 0.0 / 0.1 / 24.9 | 0.0 / 0.0 / 0.0 | 0.3 / 1.5 / 31.9 | 0.1 / 0.3 / 12.5 | 0.0 / 0.2 / 9.0 | 0.1 / 0.8 / 31.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
EffecientPairWise2D-3DMatching | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 42.6 / 78.2 / 95.7 | 8.5 / 14.1 / 20.6 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |