Benchmark

Listed below are the public results on the three benchmark datasets. The localization results are reported as the percentage of query images which where localized within three given translation and rotation thresholds, for each condition.

Aachen Day-Night dataset

Localization thresholds:

  • Day: (0.25m, 2°) / (0.5m, 5°) / (5m, 10°)
  • Night: (0.5m, 2°) / (1m, 5°) / (5m, 10°)
MethoddaynightLast updated
Hierarchical-Localization NV+SP79.9 / 88.0 / 93.441.8 / 56.1 / 74.5April 13, 2019, 3:15 a.m.
DenseVLAD & D2-Net (top-20)80.1 / 88.0 / 93.439.8 / 55.1 / 74.5April 11, 2019, 11:22 a.m.
CityScaleLocalization52.3 / 80.0 / 94.324.5 / 33.7 / 49.0May 10, 2019, 2:22 p.m.
DELF - new model0.0 / 0.0 / 0.039.8 / 61.2 / 85.7May 22, 2019, 12:48 p.m.
Localizing Visual Landmarks for 2D matching62.4 / 71.8 / 79.924.5 / 35.7 / 44.9April 20, 2019, 5:21 p.m.
DELF - old model0.0 / 0.0 / 0.039.8 / 60.2 / 84.7May 21, 2019, 2:52 p.m.
Active Search57.3 / 83.7 / 96.619.4 / 30.6 / 43.9March 13, 2019, 9 a.m.
saliency_ranking_net (multi-scale)0.0 / 0.0 / 0.044.9 / 59.2 / 77.6April 29, 2019, 2:33 p.m.
DGCNCCC21.0 / 44.2 / 80.615.3 / 32.7 / 57.1May 18, 2019, 4:24 a.m.
Upright RootSIFT (Feature Challenge Baseline)0.0 / 0.0 / 0.033.7 / 52.0 / 65.3May 2, 2019, 8:57 a.m.
ELF0.0 / 0.0 / 0.013.3 / 21.4 / 30.6May 3, 2019, 8:36 p.m.
DenseVLAD0.0 / 0.1 / 22.80.0 / 2.0 / 14.3April 11, 2019, 5:43 p.m.
NetVLAD0.0 / 0.2 / 18.90.0 / 2.0 / 12.2March 13, 2019, 11:32 a.m.
Localizing Visual Landmarks for Place Recognition0.0 / 0.2 / 20.80.0 / 1.0 / 10.2April 20, 2019, 5:17 p.m.
DGCNCCC0.0 / 0.1 / 9.50.0 / 0.0 / 8.2March 17, 2019, 11:42 a.m.
NetVlad_NN_Pose0.0 / 0.1 / 7.00.0 / 0.0 / 4.1March 17, 2019, 11:46 a.m.
FAB-MAP0.0 / 0.0 / 4.60.0 / 0.0 / 0.0March 13, 2019, 11:54 a.m.

CMU Seasons dataset

Localization thresholds:

  • All conditions: (0.25m, 2°) / (0.5m, 5°) / (5m, 10°)
MethodurbansuburbanparkLast updated
Hierarchical-Localization NV+SP91.7 / 94.6 / 97.774.5 / 81.5 / 91.354.3 / 62.5 / 79.0April 13, 2019, 3:15 a.m.
LocalSfM (Upright RootSIFT) (has partial access to ground truth)72.8 / 74.1 / 76.155.2 / 57.7 / 61.341.8 / 44.5 / 48.7March 13, 2019, 11:42 a.m.
Semantic Match Consistency75.0 / 82.1 / 87.844.0 / 53.6 / 63.730.0 / 37.9 / 48.2March 18, 2019, 11:47 p.m.
EffecientPairWise2D-3DMatching64.1 / 67.8 / 71.530.2 / 35.5 / 40.923.8 / 28.1 / 34.0March 20, 2019, 9:52 a.m.
DenseVLAD22.2 / 48.6 / 92.89.8 / 26.6 / 85.210.3 / 27.1 / 77.0April 15, 2019, 11:03 a.m.
Active Search55.2 / 60.3 / 65.120.7 / 25.9 / 29.912.7 / 16.3 / 20.8March 13, 2019, 9:04 a.m.
NetVLAD17.4 / 40.3 / 93.27.6 / 21.0 / 80.55.6 / 15.7 / 65.8April 11, 2019, 9:16 p.m.
CityScaleLocalization36.7 / 42.0 / 53.18.6 / 11.7 / 21.17.0 / 9.6 / 17.0March 13, 2019, 11:59 a.m.
FAB-MAP2.7 / 6.4 / 27.30.5 / 1.5 / 13.60.8 / 1.7 / 11.5March 13, 2019, 11:49 a.m.

Extended CMU Seasons dataset

Localization thresholds:

  • All conditions: (0.25m, 2°) / (0.5m, 5°) / (5m, 10°)
MethodurbansuburbanparkLast updated
Localizing Visual Landmarks for 2D matching84.3 / 89.3 / 93.068.0 / 75.1 / 84.442.4 / 51.4 / 69.7April 25, 2019, 8:12 a.m.
CityScaleLocalization71.2 / 74.6 / 78.757.8 / 61.7 / 67.534.5 / 37.0 / 42.2March 8, 2019, 8:20 p.m.
Localizing Visual Landmarks for Place Recognition17.3 / 42.5 / 89.05.8 / 19.4 / 76.16.6 / 23.1 / 73.0April 20, 2019, 5:20 p.m.
DenseVLAD14.7 / 36.3 / 83.95.3 / 18.7 / 73.95.2 / 19.1 / 62.0March 12, 2019, 3:21 p.m.
NetVLAD12.2 / 31.5 / 89.83.7 / 13.9 / 74.72.6 / 10.4 / 55.9March 20, 2019, 10:56 a.m.

RobotCar Seasons dataset

Localization thresholds:

  • All conditions: (0.25m, 2°) / (0.5m, 5°) / (5m, 10°)
Methodday allnight allLast updated
Semantic Match Consistency50.3 / 79.3 / 95.27.1 / 22.4 / 45.3March 18, 2019, 11:47 p.m.
Hierarchical-Localization NV+SP53.0 / 79.3 / 95.05.9 / 17.1 / 29.4April 13, 2019, 3:15 a.m.
Active Search on sequences of camera triplets (uses ground truth relative poses)46.6 / 80.1 / 97.05.8 / 21.0 / 43.1March 13, 2019, 12:03 p.m.
Localizing Visual Landmarks for 2D matching51.1 / 77.7 / 92.313.8 / 30.3 / 57.7May 16, 2019, 2:09 a.m.
EffecientPairWise2D-3DMatching42.6 / 78.2 / 95.78.5 / 14.1 / 20.6March 20, 2019, 9:52 a.m.
Scalable and Accurate Image-based Localization48.0 / 78.0 / 94.23.4 / 9.5 / 17.0May 9, 2019, 9:02 a.m.
Active Seach on Camera Triplets45.5 / 77.0 / 94.72.7 / 6.9 / 12.1March 13, 2019, 9:31 a.m.
CityScaleLocalization45.3 / 73.5 / 90.10.6 / 2.6 / 7.2March 13, 2019, 11:59 a.m.
Active Search35.6 / 67.9 / 90.40.9 / 2.1 / 4.3March 13, 2019, 9:04 a.m.
DenseVLAD7.6 / 31.2 / 91.21.0 / 4.4 / 22.7April 11, 2019, 5:44 p.m.
Localizing Visual Landmarks for Place Recognition7.9 / 30.0 / 85.94.1 / 15.7 / 59.1May 16, 2019, 2:08 a.m.
NetVLAD6.4 / 26.3 / 90.90.3 / 2.3 / 15.9March 11, 2019, 8:53 a.m.
DGCNCCC7.2 / 28.3 / 87.60.0 / 1.1 / 13.4May 21, 2019, 7:38 a.m.
FAB-MAP2.7 / 11.8 / 37.30.0 / 0.0 / 0.0March 13, 2019, 11:50 a.m.

InLoc dataset

Localization thresholds:

  • All conditions: (0.25m, 10°) / (0.5m, 10°) / (1m, 10°)
Methodduc1duc2Last updated
InLoc40.9 / 58.1 / 70.235.9 / 54.2 / 69.5April 30, 2019, 1:52 p.m.
DenseVLAD0.0 / 1.5 / 5.10.0 / 0.8 / 2.3April 30, 2019, 3:02 p.m.

SILDa Weather and Time of Day dataset

Localization thresholds:

  • All conditions: (0.25m, 2°) / (0.5m, 5°) / (5m, 10°)
MethodeveningsnownightLast updated
DenseVLAD (top-20) + Colmap Image Registrator28.2 / 56.2 / 71.23.1 / 9.4 / 36.816.3 / 33.4 / 50.7April 30, 2019, 12:24 p.m.
Localizing Visual Landmarks for Place Recognition0.3 / 7.1 / 41.00.0 / 0.0 / 0.90.0 / 2.7 / 56.9April 20, 2019, 5:17 p.m.
DenseVLAD0.2 / 6.2 / 42.10.0 / 0.0 / 0.70.0 / 2.6 / 53.9April 18, 2019, 12:59 p.m.
DGCNCCC0.0 / 0.0 / 0.00.0 / 0.0 / 0.00.0 / 0.0 / 0.0May 19, 2019, 1:16 p.m.