Assessing Thermal Imagery Integration into Object Detection Methods on Air-Based Collection Platforms

Abstract

Object detection models commonly focus on utilizing the visible spectrum via Red-Green-Blue (RGB) imagery. Due to various limitations with this approach in low visibility settings, there is growing interest in fusing RGB with thermal long wave infrared (LWIR) (7.5 - 13.5 μm) images to increase object detection performance. However, we still lack baseline performance metrics evaluating RGB, LWIR and RGB-LWIR fused object detection machine learning models, especially from air-based platforms. This study undertakes such an evaluation finding that a blended RGB-LWIR model generally exhibits superior performance compared to traditional RGB or LWIR approaches. For example, an RGB-LWIR blend only performed 1-5% behind the RGB approach in predictive power across various altitudes and periods of clear visibility. Yet, RGB fusion with a thermal signature overlayed provides edge redundancy and edge emphasis, both which are vital in supporting edge detection machine learning algorithms.