Reaching into the past: Deep learning and historic aerial imagery

Deep learning has revolutionised the field of computer vision, enabling imagery to be analysed quickly and efficiently to extract valuable data. At Manaaki Whenua we are increasingly using deep learning to extract features from aerial and satellite imagery for generating maps. As increasing amounts of historic aerial imagery are digitised, it becomes possible to use deep learning to extract features for snapshots in time, allowing changes to be detected and trends to be analysed. In one such study, we are measuring the changing amount of available public and private urban green space in three of New Zealand’s cities: Auckland, Wellington and Hamilton.

One of the steps is to extract building footprints from historic aerial images taken in the 1940s and 1980s. Our approach was to train a deep learning encoder-decoder network (Unet64¹) on recent (monochromed) imagery using the LINZ building layer². Whilst this yielded excellent results for modern imagery, performance was unacceptably poor when applying the model to the historic aerial images. First, the historic imagery was taken using photographic film rather than digital capture, resulting in a completely different noise (graininess) profile. Second, the lighting conditions varied enormously across the three time periods, with both shadowing (from varying light angle) and contrast varying substantially both across the image sets and within the earlier sets.

Finally, the image quality for the 1940s imagery was particularly low, with poor (and variable) focus and high levels of lens dropoff. However, there were no training labels available to train specific models for the historic image sets because of a lack of label data. To overcome these issues, we tried a variety of methods for dealing with imperfect data. First, the historic images were pre-processed to overcome some of the shortfalls, such as histogram normalisation, noise reduction and sharpening. We then trained a model for the first city (Hamilton), using the 1980s imagery but the current building labels.

This training set had substantial imperfections: many of the buildings in the current labels did not exist in 1980 or had been modified. Further, the orthomosaic created from the historic imagery was not perfectly registered because of unpredictable distortions in the images; buildings were often displaced by up to several metres. Nonetheless, by including only image tiles where there was reasonable agreement between the historic image and the modern labels, we succeeded in training a model that reliably detected buildings in the raw 1980s imagery. We then retrained the model using the pre-processed images, and used this model to detect buildings in the 1940s imagery with impressive accuracy, despite the substantial differences between the image sets.

Finally, we demonstrated that the Hamilton models gave excellent prediction for Auckland city across all tree time periods, demonstrating the generality of the approach. This approach has the potential to be applied to other domains to unlock valuable insights from the growing collections of historic aerial imagery.

References
Ronneberger, Olaf; Fischer, Philipp; Brox, Thomas (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv:1505.04597