Applying segmentation and neural networks to detect and quantify marine debris from aerial images captured by unmanned aerial system and mobile device

Marine debris is a global issue with adverse impacts on the marine environment, wildlife, economy, and human health. Its presence on beaches may vary due to topography, hydrological conditions, proximity to litter sources, and the extent of beach use. Studies of these parameters on beach litter are essential for understanding spatial and temporal patterns; however, this work is labor-intensive and time-consuming. To overcome these obstacles while gaining higher spatial and temporal resolution data, two methods were developed (1) segmentation and (2) regions with convolutional neural networks (R-CNN) to detect and quantify marine macro-debris using high-resolution imagery. Data to develop and test the methods were collected using a small rotary Unmanned Aerial System (UAS) with an RGB sensor at various altitudes over a 100m section of sandy beach of Mustang Island, Texas. Images were processed through structure-from-motion photogrammetry to derive orthomosaics for each flight. Orthomosaics were then run through an image processing, and classification workflow developed for segmentation and delineation of imaged debris. The segmentation algorithm detected the most debris at the lowest altitude (215 of 341 total at 15m), with decreasing detections at 22m (101) and 35m (50). The second method applied deep learning object detection to smartphone images. Based on the dataset and the network architecture, R-CNN mean average precision can range from 31.4% to 66% (Girshick et al., 2014). Training for R-CNN consisted of three stages: extract region proposals, train AlexNet to classify objects, and train a bounding box regression model to locate the debris. Average precision for the Specifically Engineered Algorithm for Gathering and Understanding Litter Location (SEAGULL) detector was ~22%, meaning it has a low performance at detecting all the debris in the testing dataset and correctly predicting whether or not that region was debris or not. Object detection has been a challenging task for decades, and with a low overall debris detection accuracy, it needs to be further improved with the use of a larger dataset or adjusting the training parameters. Few studies have been published on this topic, but this work demonstrates that remote sensing with UAS has the potential to increase research efficiency.