Multi-Band-Image Based Detection of Apple Surface Defect Using Machine Vision and Deep Learning

Yan Tang, Hongyi Bai, Laijun Sun, Yu Wang, Jingli Hou, Yonglong Huo, Rui Min

Accurate surface defect extraction of apples is critical for their quality inspection and marketing purposes. Using multi-band images, this study proposes a detection method for apple surface defects with a combination of machine vision and deep learning. Five single bands, 460, 522, 660, 762, and 842 nm, were selected within the visible and near-infrared. By using a near-infrared industrial camera with optical filters, five single-band images of an apple could be obtained. To achieve higher accuracy of defect extraction, an improved U-Net was designed based on the original U-Net network structure. More specially, the partial original convolutions were replaced by dilated convolutions with different dilated rates, and an attention mechanism was added. The loss function was also redesigned during the training process. Then the traditional algorithm, the trained U-Net and the trained improved U-Net were used to extract defects of apples in the test set. Following that, the performances of the three methods were compared with that of the manual extraction. The results show that the near-infrared band is better than the visible band for defects with insignificant features. Additionally, the improved U-Net is better than the U-Net and the traditional algorithm for small defects and defects with irregular edges. On the test set, for single-band images at 762 nm, the improved U-Net had the best defect extraction with an mIoU (mean intersection over union) and mF1-score of 91% and 95%, respectively.