Workshop on Long-Term Visual Localization under Changing Conditions
During CVPR 2020 we will be hosting the second installment of the long-term visual localization under changing conditions workshop. We will again have three competitions for different visual localization scenarios. Please see the main workshop page for details. You can find the workshop page for last year's competition here.
Results can be submitted to the challenge under the Submission tab above. Select the relevant challenge from the dropdown menu. For the autonomous vehicle and handheld devices challenges, please submit the method once per dataset, using the same method name for all submissions. Datasets for which no results are available for a method will be displayed with zeroes.
The ranking is performed using the Shultze method.
The pose error thresholds which have been used for each dataset to calculate the fraction of correctly localized images are listed for each dataset on the Benchmark page.
Listed below are the public results on the three benchmark datasets.
Visual localization for autonomous vehicles challenge
Method | Extended CMU Seasons | RobotCar Seasons | SILDa | |||||
---|---|---|---|---|---|---|---|---|
urban | suburban | park | day all | night all | evening | snow | night | |
KAPTURE-R2D2-APGeM | 96.7 / 98.9 / 99.7 | 94.4 / 96.8 / 99.2 | 83.6 / 89.0 / 95.5 | 55.1 / 82.1 / 97.3 | 28.8 / 58.8 / 89.4 | 31.9 / 66.6 / 92.5 | 0.5 / 5.8 / 89.2 | 30.5 / 54.2 / 78.5 |
S2DHM feature optimization | 62.9 / 81.9 / 95.9 | 55.1 / 77.6 / 94.7 | 50.1 / 72.0 / 88.1 | 52.8 / 80.0 / 95.2 | 34.4 / 69.7 / 94.6 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
Active Search v1.1 | 81.0 / 87.3 / 92.4 | 62.6 / 70.9 / 81.0 | 45.5 / 51.6 / 62.0 | 43.6 / 76.0 / 94.0 | 1.8 / 7.4 / 14.2 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
sgVLAD + ELF | 58.3 / 65.8 / 69.0 | 46.5 / 54.2 / 57.9 | 16.1 / 19.2 / 20.7 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
wasabi | 7.9 / 21.3 / 75.2 | 3.8 / 13.9 / 67.3 | 2.4 / 9.1 / 54.5 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
sgVLAD | 7.3 / 19.9 / 76.9 | 3.6 / 12.8 / 69.1 | 2.1 / 9.1 / 57.4 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
Local feature challenge
Method | Aachen |
---|---|
night | |
SuperPoint + SuperGlue | 45.9 / 70.4 / 88.8 |
ASLFeat (10k) + OANet | 48.0 / 67.3 / 88.8 |
R2D2_40K | 48.0 / 67.3 / 88.8 |
sd2ss_zoo4cro_supersc_p2 | 45.9 / 70.4 / 88.8 |
sd2ss_superSC_all | 44.9 / 71.4 / 88.8 |
CA-R2D2 | 48.0 / 65.3 / 88.8 |
d2net519 | 45.9 / 69.4 / 88.8 |
hpf | 44.9 / 70.4 / 87.8 |
r2d2519 | 44.9 / 64.3 / 85.7 |
nv-sp-gp | 46.9 / 63.3 / 84.7 |
superpoint519 | 40.8 / 59.2 / 78.6 |
Visual localization for handheld devices challenge
Method | Aachen | InLoc | ||
---|---|---|---|---|
day | night | duc1 | duc2 | |
Hierarchical-Localization + SuperGlue | 89.6 / 96.1 / 98.8 | 44.9 / 71.4 / 88.8 | 49.0 / 69.2 / 79.8 | 53.4 / 77.1 / 80.9 |
ONavi | 85.7 / 93.7 / 98.9 | 48.0 / 71.4 / 88.8 | 41.9 / 68.2 / 84.3 | 50.4 / 76.3 / 80.2 |
Visual Localization Using Dense Semantic 3D Map And Hybrid Features | 90.3 / 95.5 / 97.9 | 44.9 / 67.3 / 87.8 | 48.0 / 62.6 / 79.3 | 53.4 / 64.1 / 74.8 |
KAPTURE-R2D2-APGeM | 88.7 / 95.8 / 98.8 | 44.9 / 62.2 / 85.7 | 21.7 / 37.4 / 54.5 | 23.7 / 41.2 / 54.2 |
rkpd2m_5k | 87.7 / 93.7 / 97.0 | 42.9 / 66.3 / 85.7 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
spd2ms | 88.2 / 93.2 / 97.0 | 42.9 / 66.3 / 84.7 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
nv-sp-gp | 83.9 / 92.0 / 97.6 | 46.9 / 63.3 / 84.7 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
ALL_NV_SP | 79.6 / 87.1 / 93.8 | 39.8 / 61.2 / 77.6 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
Active Search v1.1 | 85.3 / 92.2 / 97.9 | 27.6 / 38.8 / 56.1 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |
coarse2fine | 73.2 / 84.5 / 92.8 | 31.6 / 54.1 / 76.5 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 |