A group of researchers from several Lithuanian universities combined unmanned aerial vehicles with artificial intelligence to capture images of building facades, which would allow them to classify different buildings and monitor changes in urban environments.

For long a tool in modern warfare, drones are now being used extensively by film-makers and even researchers. While digitised cities make it possible to observe the various changes in buildings, outdoor image processing is difficult in typical European metropolitan areas due to various obstructions to perspectives such as wires, overhangs, poles, and other parts of a building. Additionally, the dynamically changing weather conditions also affect the quality of UAV imagery, the researchers observed.

Led by Rytis Maskeliūnas from the Kaunas University of Technology (KTU) Department of Multimedia Engineering, the research team chose the Lithuanian city of Vilnius, specifically the Old Town and surrounding areas for their study, with an aim to use UAV technology to detect changes in building facades against a crowded city background.

“Being a UNESCO World Heritage Site, the Old Town area is full of buildings that vary in style, from Gothic church spires to state-of-the-art glass structures,” says Andrius Katkevičius, Professor at the Department of Electronic Systems, Vilnius Gediminas Technical University.

Such a challenge demanded the use of smart signal processing solutions based on artificial intelligence (AI) to detect, identify and classify buildings against the overall background of the city, Maskeliūnas says.

Drone assessments of buildings

“The facade is the identity of the building, the architectural face that allows the building to be assigned to one or another category or style,” says Darius Plonis, pPofessor at the Department of Electronic Systems, Vilnius Gediminas Technical University. If the facade of a heritage building, for instance, changes in any manner – either illegally or due to weather impact – a passing drone would record it.

The photos of building facades captured by the drone are sent wirelessly to a computing platform that uses algorithms to classify each facade.

Maskeliūnas believes that the images could also be used to assess any damage to buildings from water penetration, cracks, discolouration or even dents in the surfaces, potentially allowing for a fairly accurate and automated assessment.

An overcrowded city is a challenge

The study aims to detect the boundaries of a building’s facade in the face of changing weather and light conditions, and to determine its actual style based on facade taxonomy. The complexity of the task depends on the shape of the building’s facade, the natural conditions, and the urban background behind it, or even buildings with a similar facade structure, which could pose many problems; however, the drone can collect detailed, high-quality images of the facade from all angles.

Explaining the use of AI-based signal processing solutions in the study, Maskeliūnas said, “Distance and environmental factors make it difficult to distinguish the contours of facade elements, and false contours can be caused by shadows or reflections, as well as camera angle and lens distortion. Simply extracting the correct shape does not guarantee successful further classification.”

Images of the urban landscape captured by drones can also be used for visual landmarks, identifying defects in historic buildings and urban heritage sites, detecting illegal constructions, or planning urban development.

The research is published in the journal Electronics.

Image: Famous Vilnius (Wilno) buildings used in the research. Credit: Electronics (2022). DOI: 10.3390/electronics11213450