Benchmark
Listed below are the public results on the three benchmark datasets. The localization results are reported as the percentage of query images which where localized within three given translation and rotation thresholds, for each condition.
Aachen Day-Night dataset
Localization thresholds:
- Day: (0.25m, 2°) / (0.5m, 5°) / (5m, 10°)
- Night: (0.5m, 2°) / (1m, 5°) / (5m, 10°)
| Method | day | night | Last updated |
|---|---|---|---|
| Hierarchical-Localization NV+SP | 79.9 / 88.0 / 93.4 | 41.8 / 56.1 / 74.5 | April 13, 2019, 3:15 a.m. |
| DenseVLAD & D2-Net (top-20) | 80.1 / 88.0 / 93.4 | 39.8 / 55.1 / 74.5 | April 11, 2019, 11:22 a.m. |
| CityScaleLocalization | 52.3 / 80.0 / 94.3 | 24.5 / 33.7 / 49.0 | May 10, 2019, 2:22 p.m. |
| Localizing Visual Landmarks for 2D matching | 62.4 / 71.8 / 79.9 | 24.5 / 35.7 / 44.9 | April 20, 2019, 5:21 p.m. |
| Active Search | 57.3 / 83.7 / 96.6 | 19.4 / 30.6 / 43.9 | March 13, 2019, 9 a.m. |
| saliency_ranking_net (multi-scale) | 0.0 / 0.0 / 0.0 | 44.9 / 59.2 / 77.6 | April 29, 2019, 2:33 p.m. |
| DGCNCCC | 21.0 / 44.2 / 80.6 | 15.3 / 32.7 / 57.1 | May 18, 2019, 4:24 a.m. |
| Upright RootSIFT (Feature Challenge Baseline) | 0.0 / 0.0 / 0.0 | 33.7 / 52.0 / 65.3 | May 2, 2019, 8:57 a.m. |
| ELF | 0.0 / 0.0 / 0.0 | 13.3 / 21.4 / 30.6 | May 3, 2019, 8:36 p.m. |
| DenseVLAD | 0.0 / 0.1 / 22.8 | 0.0 / 2.0 / 14.3 | April 11, 2019, 5:43 p.m. |
| NetVLAD | 0.0 / 0.2 / 18.9 | 0.0 / 2.0 / 12.2 | March 13, 2019, 11:32 a.m. |
| Localizing Visual Landmarks for Place Recognition | 0.0 / 0.2 / 20.8 | 0.0 / 1.0 / 10.2 | April 20, 2019, 5:17 p.m. |
| DGCNCCC | 0.0 / 0.1 / 9.5 | 0.0 / 0.0 / 8.2 | March 17, 2019, 11:42 a.m. |
| NetVlad_NN_Pose | 0.0 / 0.1 / 7.0 | 0.0 / 0.0 / 4.1 | March 17, 2019, 11:46 a.m. |
| FAB-MAP | 0.0 / 0.0 / 4.6 | 0.0 / 0.0 / 0.0 | March 13, 2019, 11:54 a.m. |
CMU Seasons dataset
Localization thresholds:
- All conditions: (0.25m, 2°) / (0.5m, 5°) / (5m, 10°)
| Method | urban | suburban | park | Last updated |
|---|---|---|---|---|
| Hierarchical-Localization NV+SP | 91.7 / 94.6 / 97.7 | 74.5 / 81.5 / 91.3 | 54.3 / 62.5 / 79.0 | April 13, 2019, 3:15 a.m. |
| LocalSfM (Upright RootSIFT) (has partial access to ground truth) | 72.8 / 74.1 / 76.1 | 55.2 / 57.7 / 61.3 | 41.8 / 44.5 / 48.7 | March 13, 2019, 11:42 a.m. |
| Semantic Match Consistency | 75.0 / 82.1 / 87.8 | 44.0 / 53.6 / 63.7 | 30.0 / 37.9 / 48.2 | March 18, 2019, 11:47 p.m. |
| EffecientPairWise2D-3DMatching | 64.1 / 67.8 / 71.5 | 30.2 / 35.5 / 40.9 | 23.8 / 28.1 / 34.0 | March 20, 2019, 9:52 a.m. |
| DenseVLAD | 22.2 / 48.6 / 92.8 | 9.8 / 26.6 / 85.2 | 10.3 / 27.1 / 77.0 | April 15, 2019, 11:03 a.m. |
| Active Search | 55.2 / 60.3 / 65.1 | 20.7 / 25.9 / 29.9 | 12.7 / 16.3 / 20.8 | March 13, 2019, 9:04 a.m. |
| NetVLAD | 17.4 / 40.3 / 93.2 | 7.6 / 21.0 / 80.5 | 5.6 / 15.7 / 65.8 | April 11, 2019, 9:16 p.m. |
| CityScaleLocalization | 36.7 / 42.0 / 53.1 | 8.6 / 11.7 / 21.1 | 7.0 / 9.6 / 17.0 | March 13, 2019, 11:59 a.m. |
| FAB-MAP | 2.7 / 6.4 / 27.3 | 0.5 / 1.5 / 13.6 | 0.8 / 1.7 / 11.5 | March 13, 2019, 11:49 a.m. |
Extended CMU Seasons dataset
Localization thresholds:
- All conditions: (0.25m, 2°) / (0.5m, 5°) / (5m, 10°)
| Method | urban | suburban | park | Last updated |
|---|---|---|---|---|
| Localizing Visual Landmarks for 2D matching | 84.3 / 89.3 / 93.0 | 68.0 / 75.1 / 84.4 | 42.4 / 51.4 / 69.7 | April 25, 2019, 8:12 a.m. |
| CityScaleLocalization | 71.2 / 74.6 / 78.7 | 57.8 / 61.7 / 67.5 | 34.5 / 37.0 / 42.2 | March 8, 2019, 8:20 p.m. |
| Localizing Visual Landmarks for Place Recognition | 17.3 / 42.5 / 89.0 | 5.8 / 19.4 / 76.1 | 6.6 / 23.1 / 73.0 | April 20, 2019, 5:20 p.m. |
| DenseVLAD | 14.7 / 36.3 / 83.9 | 5.3 / 18.7 / 73.9 | 5.2 / 19.1 / 62.0 | March 12, 2019, 3:21 p.m. |
| NetVLAD | 12.2 / 31.5 / 89.8 | 3.7 / 13.9 / 74.7 | 2.6 / 10.4 / 55.9 | March 20, 2019, 10:56 a.m. |
RobotCar Seasons dataset
Localization thresholds:
- All conditions: (0.25m, 2°) / (0.5m, 5°) / (5m, 10°)
| Method | day all | night all | Last updated |
|---|---|---|---|
| Semantic Match Consistency | 50.3 / 79.3 / 95.2 | 7.1 / 22.4 / 45.3 | March 18, 2019, 11:47 p.m. |
| Hierarchical-Localization NV+SP | 53.0 / 79.3 / 95.0 | 5.9 / 17.1 / 29.4 | April 13, 2019, 3:15 a.m. |
| Active Search on sequences of camera triplets (uses ground truth relative poses) | 46.6 / 80.1 / 97.0 | 5.8 / 21.0 / 43.1 | March 13, 2019, 12:03 p.m. |
| Localizing Visual Landmarks for 2D matching | 51.1 / 77.7 / 92.3 | 13.8 / 30.3 / 57.7 | May 16, 2019, 2:09 a.m. |
| EffecientPairWise2D-3DMatching | 42.6 / 78.2 / 95.7 | 8.5 / 14.1 / 20.6 | March 20, 2019, 9:52 a.m. |
| Scalable and Accurate Image-based Localization | 48.0 / 78.0 / 94.2 | 3.4 / 9.5 / 17.0 | May 9, 2019, 9:02 a.m. |
| Active Seach on Camera Triplets | 45.5 / 77.0 / 94.7 | 2.7 / 6.9 / 12.1 | March 13, 2019, 9:31 a.m. |
| CityScaleLocalization | 45.3 / 73.5 / 90.1 | 0.6 / 2.6 / 7.2 | March 13, 2019, 11:59 a.m. |
| Active Search | 35.6 / 67.9 / 90.4 | 0.9 / 2.1 / 4.3 | March 13, 2019, 9:04 a.m. |
| DenseVLAD | 7.6 / 31.2 / 91.2 | 1.0 / 4.4 / 22.7 | April 11, 2019, 5:44 p.m. |
| Localizing Visual Landmarks for Place Recognition | 7.9 / 30.0 / 85.9 | 4.1 / 15.7 / 59.1 | May 16, 2019, 2:08 a.m. |
| NetVLAD | 6.4 / 26.3 / 90.9 | 0.3 / 2.3 / 15.9 | March 11, 2019, 8:53 a.m. |
| FAB-MAP | 2.7 / 11.8 / 37.3 | 0.0 / 0.0 / 0.0 | March 13, 2019, 11:50 a.m. |
InLoc dataset
Localization thresholds:
- All conditions: (0.25m, 10°) / (0.5m, 10°) / (1m, 10°)
| Method | duc1 | duc2 | Last updated |
|---|---|---|---|
| InLoc | 40.9 / 58.1 / 70.2 | 35.9 / 54.2 / 69.5 | April 30, 2019, 1:52 p.m. |
| DenseVLAD | 0.0 / 1.5 / 5.1 | 0.0 / 0.8 / 2.3 | April 30, 2019, 3:02 p.m. |
SILDa Weather and Time of Day dataset
Localization thresholds:
- All conditions: (0.25m, 2°) / (0.5m, 5°) / (5m, 10°)
| Method | evening | snow | night | Last updated |
|---|---|---|---|---|
| DenseVLAD (top-20) + Colmap Image Registrator | 28.2 / 56.2 / 71.2 | 3.1 / 9.4 / 36.8 | 16.3 / 33.4 / 50.7 | April 30, 2019, 12:24 p.m. |
| Localizing Visual Landmarks for Place Recognition | 0.3 / 7.1 / 41.0 | 0.0 / 0.0 / 0.9 | 0.0 / 2.7 / 56.9 | April 20, 2019, 5:17 p.m. |
| DenseVLAD | 0.2 / 6.2 / 42.1 | 0.0 / 0.0 / 0.7 | 0.0 / 2.6 / 53.9 | April 18, 2019, 12:59 p.m. |
| DGCNCCC | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | May 19, 2019, 1:16 p.m. |