Image capturing, segmentation and datananalysis of shredded refuse streams

Refuse sorting is an important cornerstone of the recycling industry, but ever-changing refuse compositions and the desire to increasenrecycling rates still pose many unsolved challenges. The digitalisation of refuse sorting plants promises to overcome these challengesnby optimising and automatically adapting the sorting process. This publication describes a system for image capturing, segmentationbasednrefuse recognition and data analysis of shredded refuse streams. The image capturing collects multispectral 2D and 3D imagesnof the refuse streams on conveyor belts. The image recognition performs a semantic segmentation of the images to determine thenrefuse composition from the 2D images, whereas the 3D images approximate the volumes on the conveyor belts. The semanticnsegmentation is done by a combined convolutional neural network model, consisting of a foreground–background and a refuse classnsegmentation. Both models rely on synthetic training data to reduce the necessary amount of manually labelled training data, whereasnthe final segmentation performance reaches an Intersection over Union of up to 75%. The results of the semantic segmentation andnvolume estimation are combined with data of the shredding machinery by transforming it into a unified representation. This combinedndataset is the basis for estimating the processed refuse masses from the semantic segmentation and volume estimation.